{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "# CRNN-CTC 验证码识别"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "😋😋公众号算法美食屋后台回复关键词：**torchkeras**，获取本文notebook源代码和数据集下载链接。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "本范例我们使用经典的 CRNN+ CTC Loss 的OCR模型来识别验证码。\n",
    "\n",
    "我们通过导入一个叫 [captcha](https://github.com/lepture/captcha/) 的库来生成验证码。\n",
    "\n",
    "我们生成验证码的字符由数字和大写字母组成。\n",
    "\n",
    "项目参考： https://github.com/ypwhs/captcha_break \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "#!pip install captcha torchkeras "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-18T11:19:45.698786Z",
     "start_time": "2019-06-18T11:19:45.381128Z"
    }
   },
   "outputs": [],
   "source": [
    "import torch \n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "from torchvision.transforms.functional import to_tensor, to_pil_image\n",
    "\n",
    "from tqdm import tqdm\n",
    "import random\n",
    "import numpy as np\n",
    "\n",
    "import torchkeras \n",
    "from pathlib import Path\n",
    "from collections import OrderedDict \n",
    "\n",
    "\n",
    "characters = '-' + '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ'  # 注：’-‘ 为[blank] 特殊字符\n",
    "width, height  = 192, 64\n",
    "n_classes = len(characters)\n",
    "\n",
    "txt_length = 4 #识别的验证码长度\n",
    "seq_length = 12 #CRNN输出序列长度，一般要求 seq_length>=2*txt_length+1 \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一，准备数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAABACAIAAADkhTlJAAA7wElEQVR4nO29WXMcR5YueI57rB65AEhsJAjui0SKkiiJVKkWqbqqp62rzdruyzzPPPcvmJd5ujYv8zL/Y8zuzLW503eqV1VJqpJKS1GkRHFfQIIAiCWRW4RHhG9nHiIBkhIJghIpUdX8jGsC8PT08ONn+85x7Lo+PMdzPMePE94PPYHneI6toGWWtVcXz36285U3ktaEL5IfekbPFtgPPYHneI6HopLeq+/+VrZXF89+9kNP51nEcw38HE8LWmbVP76L2tyU252vvPEE5vQXh+ca+DmeCirleeXd32bt1U1J/hbY+cobojVx8Fe/eW4/PxD4tSDWEzk1n+PHju+4DTZNXwAQrYlDv/rNt95Ozzfk1rhPAz+pU/M5ftR4ItvgSZm+vkiqX99lkL9g3BXgZzZgoGVW/fqhJ/IfAk9qGzw3fb8f3BfEegYDBvdlEZ7bUdvDdzQ7v/s28EWSABz61W++9RyeY5u46wM/gwm3J+hK/UiApshVnmmZkXOAAABA4IvEj0Wwvc+++Rynj73ix4KIADEQtUAkzPMBaJs//uxsg+fYAvcFsZ61gIGW2ZV3fyvbqwDwF2qMoVWlyjKdp85aBDSlunPhc9lZIwLYkGA/EjOvvBEkte2MqHN586P3lMy47zd37UmXlwhg6ujLEwePhY064952ZLj6x1/cav8F4j4T+hl8YDtfeeNpaINv7FFEpWyaKSkdY14t8YVAjz+pt3sYENAps3z+i/b1S85oRNRloWSmhhoYEQCIfJHI9urGVOm+nwcEIhqKOiCAzqXOpZYZ87x0bQUcEZBRpR+K6SMvI3+U+D79baBzqYZBDUIczocIAIAx5sdJIBIvEo88aB4ENGWupNQy9YLQF0kgEkB8otN/tvADEznIOV3kCIiMeVH8tWf2lFypTStx5pWTkbEMQMnMZpKuXVm5fkuG4fjhw819+8J6zRMC+dNNldtSFd31/tJtU+TIEAAJAIiqTYcAgKhlpvMM2mv3rg/hhtTSpqqufma4Ya3RQAhAzlkzSCEr0DoA/FaCsTXQaVVmqdOK+UFYqzPPBwBTFlpmSqbkbHXYVDCyWPji03zQ88ABAPOZMWQJOGMIEDfHdp98Ox4jPxaPPQ8AW+iFLz7r3rwRJ/U9b73tC3H3jZ8BPHHr5ocUYHJUpoPewnxn7trkwaMs8hnngUh8UeNBWO2zJ64NNv1qlaXZ2sqek2+rTq9/7WJZdIqssLKwnKf9ldG5MzvfONHctw/5GEGw/U1P5FSWmSJHxgORfPNUeiDubjHafIHuSuZQ6Kj6KiIBI0QgQnCseh1xKO40/K67P8wAldZ5XuRZHoYe409wQ6OzShdZ2e+vXj7fmbsxcejY9Asve1FkVZl3ukvnTw9WFwkAEYiIcQSLalMDk2OIBESIBAhADDFfXy/7vYN/9RvRGn/Mp4+mKPLOeu/WXG9+rqjVW2tHgkYzrNXg2ZDhpxGR/UE1MFHWXp3703smSzs3r/lRDECNXft2vHwqGmFB4NN31BVEOs8BABF5FJV5qvIsX1+b/+P7sr1ayswbpJf/7b8zAFOkWpUEQIBote6VVnWaK9fE+H/yR97CxzKlHeTra7c++dDz/eljJzzBA2EDkfCgQVB/uDAPhc4RATgCxxAcIENOhMiILHBH1vpA1vkmqkXBiO/BFBBz1qosVeVAmxSZ434YMOEcs8papZABMHSOrl69RpM79jYPIfe+68IOgc5pJQcr82fvnDtdLuZWKcqLWNRGdx1Yv3Rx8eLptLemc+mcAagsWSQAJKwmgMirUwoBccPsUFnqRfHC2U8ry2s70zBloTKp8szIYvHMn+TaCpDTqrj91WkxMRUktWfBiL43Irt49rNtf7qtBvRF8gMKMBI5k0sjZbq2wjgHAu570oBrTh0ePfGtxy1krvJM55KTVYNB58aNiUMv8ChM+2vzlz83qx0aZKooAMCo3JQ5IAICIkNCoMqlJGvl/BeCJ7Ll9b36KLLtbQEinXWz1cX+7evWUm/xdtBIJvbq2ZenIfj7rVej+hPJQRREceAJEYV+7nyP+agJG3nQDrnG1TBKoA87in1HfxOEdXJIjtLVtcVzp4u4DUKNlGE4uZ9Sb/mrL2wlHshIlSAzzCQYC8ETeehIVuusv3Lzz/Nn/lguF7YwYEGFYdbvBu322uJcd/m2s5oQh0YsAW3I8IZ7f9ekIKTqtUAkgUhmXjm5zWlYVapu79bnf2ovLJNKXZ7pPAVyNpc6HWiZAdEz4gY/wTRtdRx89n/85x+6mIE2whcAAOScazTq09PTjhx/HJ62LvLKKiOiUuaXzn4ZFW1OVuc5EOvOX/ODoMizspC2zMECAGBlvAEROeb5UdII/MhIW+R9S4XRQd73lk/Ph7UDI0kT2KODtwBARGW3vf7lH3RZGKW1zKjPTeOAK19jdbP1MlR/OoCIR7Ov/Wx0djcxjoyRQ+SEDhiRtXwXgPFMLGJPJGHUBCByToyON3fPKJMyZj0/YpqvXLnIPB8oQ+BExAhmybbIPSHdC86acpCuXf984fQHZUdZZYAYoAWGnNzi3PV0aYFZU50gmwF1hkTgAJAco0qiEQAcAQIBASFjQVLb/7O/FuOTfhw/7N0RUCldFpkuUpfnK+c+W71ytUwzsg5Z9RycH0VxJII4AUSTZ9Xn9n7QMO2TjchWx8EPLcAAAICICEhAHvNEJGIh/MAncA/85mEYABGA6zxTMkNyuigWvvpz2euioyIvXJF3ZcbBDv3CYagWAJGIAZAjBx76ouYL4fGIMZzYu7/R2rNy+Yvly+dAAyJaVWRlt8wHZB1uc53IlVkuM7RFDtxjDOtibFocwHAHwUPCqgjEAHColBCBPC8UY+HozqiePEpzEAAgY2GjHjbqG+MxnWWc38CheAACMKCB1WPO8O8cwiLrlJRqkC7fOL9w9mM7UFZpQA7ouO8HQhD3pspMp2mmDfgciBiQIeSx8BPRbHim9J31LCPGCQhUmZlU6jwDRGQYjY4FSS2Ik4dtgCG0vX3panvxc+ytm06ppQSyiOSFURALLxYQBbM/eSdqNhFR53Lt84/jXft8Ua80PI+j77YMj40nHpHd+cobi2c+/YEFGBHZ3TAvAmPEtvI47waQXz3FWXj7yy/TO9fBWV0WKk9tIZ11VfiGAxAyRmAZ96PEj2t+xMuyBw6RBUGcKCqiHdNT00esLsHqWnMsiGv+2LRXX3D9VWcckSOyjuz2tRYBOuSWcQcMCRCZNzXJDh3yAn/LzQh3g1eI6PFr16+/smdvDNt9zERW5amSmc5zJG6VkoOOcWojUo0GKLUyLdOalGEY4LeKYznnCpnrNO3enl8+d0b2lm2eG2URGfc8P07CZjx9/LWx5u728hclETK2cYC4mPuqMbnvpz9pNEcQ0DkEBELQeZat3rn50XsAOLT3OQJ7pPGFVinoLhfX5nWecuDgkIi8MIwaYvepX8atqUAIXyS+qLk8K9sr67dvyYsXuKg16pM73/pJPDa2hYZ/SniCFR3VcfDX/+v//nQFuMgLnUsAiuKIiCpdGMTVJCxCm7F1xD4OH1i13xDo7va6J1dC94UBznw2ffT1srPauz3vrEIGVQAWCcgh831fCK+eBFHELOa1iT3795Ep1levDLKbs/tOMh4YXfp+rdvtdhfmQ5OtGWztPTTSmi537FhN1x1YQFR5oaRUMot8/4Fu8H0rS2Ty1BTZ0K8Gcoy5MCYRUxjAw/QJETg3DOAgAKFK5e6RulxatoOUcUJ0gGAtA4aIhOAcEdLQtaThO1HaW1y6ctasEzJtCVVeKOMAqnAREgNnTffKtebUbNhowDboHPfDouuYVLZvL1799LI/uFNF2oEBJ+SMR6IxcfT45IsveyI0A5nZTDplqzQ1kB/GELD9L73I/dgBQwBg1WnHwOLa5UvDMARgEAkeCP/RoXvyAz8OwsiLCErnHBB5QYwhnzn1s+buWTE6zfhwb+tC9s9+lK2vZKnE9XaRrGdl99Av/xZbLS/6vvXwt8DDYtfVP56SAKNWmc6zdH390ufn6pjvPPoqEa1cPTd1+JVKgBEc0pzu/RdyI7Rh6JKyutPvLy/meRfQAQESeuARggGj8+zmh++V2cBjbPLNn3MyYRwEIi4z7cgFcczjmh8kPmPM+Xz37h2H9pBWZVEAgSqK1QvnVL6cTExo0+vfuk1t5sDKMtMyzcrcY57NSnqJgprPQuY0MULOeWfu+tjeA3Fz9BEri+hHcdnvrp7/1CpVxWWg8vG3XCkeRH4U+5Fw2pAjAPAYds9+Wd64nRMQs41EFUpr2zAAcawRXF5qRhAzLH2PSPvOd87lRV+lmS4KcIgIDhhWh0IVJgK0WqkyK4vMOeuB9zjii0CgssXB7f86957QvUGpCg4ACJ7D0PMbYb2151hy8JUwaRKj1fbp1f41zczwQCawzk4ffrV945K98hUnj6MzzgJxtKzMlVFdk+eIgAyi5sjOY6/xMHiE+AKgz1sHXsxWlqm4zIKA1RJn/InDB7qrV8JGww8bQa1eOSVFnnV1KWUKyIiolH1adld//0973vxFPDYWCOFFyVPIjT8ZPDJ2/bQ0MEO+fPOz1a9umuW1NlF2Z8k5Bwi9+dthXCcgBhah7ZRQMkWIoArhoOus3Bp8sMajENAAgq98ILQ+AJJKB6ZIrYOpE6fs0pw4+NKu13+qiqy7MFdmKTk7c/zV2uQOBkwrBQjkyBq9dP7PpErVL1SWOVO6Ug8WFpx2Ji8QGDEgAgZotdImVUr2ApuFnA8YAJGzY/v2B7Gge3gTD17ZX/8dkctlnmW6LAqOgIwHoubXEj986DFPQCzwJg4dz9ZWy3RA5IChzYsBlf10DQARXcosAlp3B5BJ5gCJiNDBABGQNrK9SAQAxIBV/+cbBg9tmDREJP1CsfJxNyuRU9mgv9C+9kFp+m0oY4bD3FbE/ZE4qh/eVe4d7926MNrZ0Zgez7pZKXMAQoYARECMhysXzjkAUxRIDpyD6mMAH57cDIEIGPNqNV8kfiQe4QAD8SCMRkabM7NBux/PzODoqAFcuXymzPv5jXf3/8Jrzu4N6gkgAovWKbIowJbVUiiZ9ZcXLv3b/9OYntpx/CfxWMsXiR9uK2P//WPr2PXTEmAC12wcWCmu2Cwl57J0sPmlHNdwGHIEIAQIh19AsGSs7IHsDcMzMPybqkQDAAB5YbR8/nT807chCkJRnz3581IOlMwQYbB0vTW7l3Fv9avP0l4PEMssVUVmZI4IQEgIKpMw5DogYRXaAvT9sN5IWuNickIrnUXLEJSu1DyKfCF8kSBj33y6X1tZIrDE+hRV/EcErLUmpw8eZ+EWCW3ywjAQNYwEMU5gkJCQgDmo1oYAhvayAyAgAreZlcENHT+0vocOx/CthlkZqCK8CISW65y58qHG/ENmqItB/87tqx+cLrrclCEyB0AYYJDUx/1kfGr6z7v78wv/58FrO/yD1ov8pBCRFjn1YBiAZqqQ93PFKgMaERzczTAhETPgb3daQCzwxvYf0Y0Jd+HCnYtfdXSmZaaylAfh9X//3d6/+ZvGrtk4ETwSu6ZfXyoD7vpUplpmKpc6TY1MtZT9Oyv12dmZIydYGBD3/DgJk1oQBE8qXP/dsXXs+ikJMPm+8INaENY26IC4kS4aihJs5o+Gu3Bj722yeivvjVWZpk23GE1ZEriFudPR9HRTjOIGI9jkUqcyX1vNzl9ZW5orVArDQStLEgHvalFkzI9FUKv7YUzk0MORPfsn9x/jQowZ1ygGd4os7fRMWWgpdZ6xb/jAOpcTh48unvl0z1vvJK0JPxbOWiSqUTkAYAjIWBAngRBeEG6tT2hIadj0gglhOFuqAujVJ9kIqG+K6L3/r5TtxkeGu1LNAAiYF4RRbXSsliR1ZI9nPxtTLl7/1OTa5Ao4co/7IgrGotE9+yZHDq5nF9lXF8ZW+KBcWbjwWU6KclLgOeCc7OYhPDyy78rxXfL2MJ8EzI+TOAj97bKgiQeBX2v0Ll5eXJ5Le6vKGkIABFsWKax99fHHL9eTKImjONn56oHxw9POlTrrLp79SEurZGbyTKWDijqe3b4dBGGGLJnef+SFo3ykyYWg4DEYeE8Jj4xdb1eAvx2Hk4g2SbsEWKkCLxYWGBeRVXkYhSaTZMgUBbBhch+IEMgREjIAh2ARyQEjYojACI0qVFvrgYQmdK9fdVohoANUWaHTfJBJR0S0oYIYCyJR5Xt1ng9f4mFz575dr/7Er4UEjvl+ECeBSLgfajkYmTnUXVhI17tWq5WLX9Qmp8Ja495UsJaZktnimU93vnoy2OgXgQAcHCfLiQDAIToAR/jgpbkHeJ8apU26UpX+2jigNjf7xoF3j0nPGQvimp8kLIo2SV2AG+Qn55DziQOHW3sPhvX6Y4agyfPC6X1v9q7/C8a+tWVcS1r7Dkwffx0Z683fWD0zx7I4KY2zYGy2eu7MatLinDU2ztyKGkPomKtmxagKwjFEvFuGAeDiWn3HsVe8aGsHGI0xpZRapqC1TeXK6lwvXTfWVCP5UQwcIYy5J6ul4CEXYV2M1QFAybHaxLSWWdHvL579VHbaWspyMNDpwAERD/R6e37h+oFjJ+H4cRY8zjo9NWwtcdsS4O/C4aQqCEoYCAEAnkj8WIy/9PqdlfaOmbFB51ZEtaXP/2xIVo4bZzwKkiBOWBhKCMGjBnQdo9QEapA5OSBrwTEnlRmUOstG9h5Yu3W12tC6yNcuXJw6drT8su8XiCJmkUDGRb3pt1q3znzspOSIQI4zEQZTRIYoAAbOmnzQVbkMRQ0ddhbms/UOIDNlUeZSSemc28xu3ev9rl4+P1LFFYiUzHQuTVFUtGQi1M7HbRzhXhDGURTGorSanAMg9Jgfi0AIP4gdAKuC7JUNbV2Zd1Qptcw5hZU0M8DRicnJ134WNOsE7u57DmWIuB9WB81GSfD2FQv6UX1keveBv/r11dPvNgwkE7O7X3mLBbw9f+X2F5+qbma0A2SIYLXVJo/zRUR0iJwgC0MQgkaTlrKhsxwj7SxWCUN0SuU6L5SUQOQ42AR4Ej/cAUYyRmeyGKS3blzrzl0NtbRWaZk5stUn9cIwaozsfPH4yuqN8am9pt/N/DBIaoGIYUjzqgeiXj2s2uR0f+n2whd/LrrrOs+oLNEZN+h1tV3qrsyUGhPx7BjSD8OjBfg7cTg3jGRCImDTr57M2qv73/oFE/Xpo8e1yqcPvLh6+RwLvCFXn8gBipnZ2ZdPefWaBUB0HK3WOuumC+dOD9J+tScZ8sH1m+N7Dnlx4okEkAFanUs3ZvyaOLb7xYWy23rldS8WiMyL4n57NZq7UpQllCUgt1auz53u37mEESMfGePOWi+MZo6+NrpzL24ouqEKpK8/xfu8X0QAUDIbLN+59fEfqxA0ADAkn5l7NMzDVohY6E0eOZ62V8tsAOAqo9lrsukjLydj00FS32AeVkvk2iu3Ln/2f7HcAzfMIhHnWBupT07HExOPDP88xuMDAABEDGpibNfuE63/0SgVhomzrr90dfHz35c9aY1CBs4RC3z0QrScVAGuQM6BKPa8cy+f+unuPQc99IYHC25SG+X66s2P31dSAiA6sIzjw2eHAM64/tzNW2dOdztLNk+lVog0JMAAlGgp5LMnT9XqE42evH7tvL5ymXuNXa+9Wp+aGpYWbnykIKlVrbbq0zO9hRvL5z+3pdVSqmyg0GkEx5510a2wLQ38bTmcyBAqOwkAmRDL64MTf/13URxXlWKxSJxWQSAY41WMinMb1qG+I2runfBrUwQWABCwyMr5hdP9fs4YB+sAwDlX37+HR5EZDMhupJwYOc58kUQ/fesgoosjR6RkVmQDsGZiZs/txQUCAHJG51ZLSNuACAwBgZylKPEnZryJmea+I4O1O8WgDwCmKNRGee2mG3xfXCEWRFQM+jf+9Ie807YqR0QihwAMiW2xJYcgL4x8kTBRA+6BHfL+dcd1bl1v7Ttan57ZEF8AACIyAQ+uvEyDy1QYAA8QCYEYPPKwqBazLErrLACEUcgeTZkAAEBkcdKMkyY5pwaDlfkL1z56lwbSaQ2ADhwGni+S8QNHTZ6tXr5oHatMD2fhHWsbhET8rpN+T5qfXBVmwyistaAVxA/L6KAzpuh1u3duDpau2rIEBjgMsQMBeWGYxOHgwEQwPSWodl0NsqWVXGk/Sq797k483tp14pQvapUYB7EAAEQMa/VAJEEiRnbtMzLvLd1eOvtZGLCkGbONerhnHNsS4G/N4bTgVSxBAlBFfujwfiD4Wp0nI+JueN4RYTI2aO1adg/aihuxaAIAB86BtWC8OBZBLY2TMh8QOSIgEbs41IXUa8v5YHDz7KdYFuDQyMzzPG00DBMwG1qNyPN8HkSslsxnqa/1XlEPkhr3fau1NXr5whf1yemw3qhM0gfGFXSa2Sw1uQS+yV8E2IYDPPxozoIrkVw1KTJUFFkgsMzy+v1cfARM6hMvnfrZ5XzFrklT6qH/bJ0a9MkpVUhgWAWv7oYO7/lxXejbC0vTe3dP7toFDLe/Tck5labrN6/c+uPvbF9arRjjgMS5z2vJjldOTe06dOfyRT8UTneAGAGEVhfnTpvrlzqMASNwFS/aAQASU7k0eRVboajWnDpxIgjjh1mtCEy2l3sLc8AR0G0E7QgQwDrGvcmXXz925FjcHFOd1FpbOgfgTJmaItVFdnV9jceC1RuH3voltsb9aLgJkbGw1gxrTbIuGhkL4rjsdUYOHmfBj0MFP1qAvz2HE4FxBxuOmEdu5cwnjbGWjqN7xkEgh2aYnESkbJC0V8drO+8bift8z769dnmu3btTPTLG2cqNSyN7DiS10ckTr/b7S2U+gGoIAJWnNz/6N9nulQOp8kyXksGQbg3Iqogtcs+LQkRyRpHSzdl9rD4yNTU91RyxumRD6/nBSu1r62DTzGUSe3nFMiIiIAIgeLT6HYIIrSw3BRWBHHiFQwND17tqsqFyCQBEDrTeue/E/Mofq2CVs66Qg87cld7cdU2SWOgAOVb9LoiGvT2QHCFCWRTIvCIJypFm3GjC9gqtyJHKss78tbmPflcOMqd0ZTcxj0dJbfrYazuPnUKAIObMc7CxfFqXultCZ/2epRgmh6GqIqyCioAsRl8ITzxUgAmcGJto7Nw1WF4AcoRss8jJEjAeNWoTY6M7vDAsCufXp5qjazpLTSEVOJPnOs8A0E+Sq0ruPfmOH8eBqPn3tP5AzoJabfzQUQBgnofeDx+C3g62pYG/BYeTAPw49qIoEEJlA6git7H4mheNzAWJ8uvkF6XWvrU8T5Oi31Aq8JPNbAn5gR/HIooEZ74DDUBk7eS+I34UcRF5QvA4QURy1TuDyTPZW+/NzxNxAGJ4nwqrxkRnRWMkHhOdm1eM5eniUuvg2MjoFLfK94OK8q7Lknve9Isvi9Hxh5akEenBoPvJZ6XL730Pck7lWVlkvq57/lb2GAIQMhONGZpjGxldhk6aQnZWuwsMAU2eLZ79VEnpAAgcc2TSnPEA0CAQAaXLS9nyAhijisJVZRFD+dgISg9TOVAZ7UuffRDFAdu9P6jVcWtDmsjIXKVp5/bc3GcfqF7PqhIZIyBkPBL16aOvzLz8ZthomFwCOEB39yC6f903E2V3hyZgiI4QrbBbuQDEPC8cGW3u2Jmv7c0GXaMKlRdOGwLywojHDU/UGWMAxKJg9tQb+uAB/OKLuc5cN+2rPAMDiGBymS0tX/63/zeIhWiN7n7jr6JR3Gy6wDyPeZsS8SOQXnia1UjEo3DnS29kqysqSwGocj82vWhyZKQEskHsdhwfDP5QB0UImIJ3m8ImIQDc6w8RkIcBIhvWoG2wZ6uvMjsM91Zp2zARIqpnoqakBIRA1PwwRkJdpDrPCQkAkWHRXy/SjjW+06V2xerNCytpb/eJN3ft2dvaf7S/vJR1O498jDrLsrXVdtbWhdzQl1TxwPrrdxbmvjw0Oo5+tEU8k4D8WLBGA0UDBgrIAQBwqNnu4p//sBLGPmNKplpmWsqhEVrZK7AhE87pDVuUGLtHPjbDXxscCgQi0GVBXbr58e99URtL6ls+RlJZli4tLX70Uad7RxepKUq2QRRFxkb2HJh68bWg2WC+T5Kscc7SMBlGgDBk62xa8n4U+0IEcQI9LKU0udRh5tdqeXMsiGtbrDYBoIfNvYfEyHi5urZ46czKjavDL3B/+sUXG5OTwBgB+XHkxxGKxLTG9ndXb1z7ILuTBdRXqdS5LCUpmUkGsrNupJ59/Rfx6KgvBI+fUSbW1niKxQw45OfjRtqexg+/qPMsawO0V4Go7PQGN26MHp7IizcBPgHggBhadTNtt5eX/lquK6SKj88IVCrT/h1jiqHHp/Wd82drE9NRrYkAwIdNWHWeL535dN9Pfzlz6pfGflDkKSBEjcbkvmNyrb34ycfOEXJAQGdcnuYOyEMEYMbqctDlRmbzzazZQOd4GHHPt1otX/iydo8PfC+UzOTa6q0//qHQ0lX5VarC1khklcxsPzWqCOERDSV56O05fESv3e6mHc9ZRPSATJbp7J529pVAwJD5AZsyCfd9y/2gDbd66De46vhzkOclpJjL0jnH+UOqDImUTPtLC9fe/de8t24K6cBttqFDxsPaiGhNRiNj3A+sLnWWKplarYDACwMMkziuM7KFHBQyRee8SMSjo3t+8g6LPL2Ul/91/UbyJwIizvYc2f+owC8xzw+bo74v2K0Ff60LjJM14GzosTAO/UQwzjYPSoojHkdRLTww9vc6z8Csg/y/r3w47sjqXBKBlrJ7+1be/m+jk7snT5yIxkY9Ibww+nGJ8dMUYGTrVy87azd32OKZT/1YDDOazpm8AGArt686ssYIwAIBgczEwrVgfelCGIYBOUKukCGZslekOVClapnJC5PnWmau2qEVyRVB55kpCgIU45MHf/0blctKTxmZrd+8mjnLqBIvYNznQeQQyShXlowRAiFR5+oF3e22DhwmIGBoslLlwzaR+I2NrmU299H7g3TF5BIQEdGLYucQkXSRMW35IMN+hqJO0UN9KgIKwyAUwsQjhvncqvvOiSFz464lXL2Imy9W9dQARI4x7oskEDUvjKp8EhFgVVeJaAEByDlHBE6jGZn2680HskQ3UfR7cx+/n/VWbVkAInNAFf0RAZE1d+wa33eE+RwASNv1m5d783NkAQCRezMvvTIx/oIZ5Lc+/6NymQNCxmZeeTMcHSe0C8ufDCbbLAMLwJxdv3FxbNfsozcVIGnVVrKjctAaEbgXeiL0k5hx/s2P4QeJHyQwAggjSv7Dwb+6c/ndj1zbmSJ3REqmWuYqz7pr8/XZPRNHjns1DIQL45rnNwhqz74wP0UBJnJjBw+v37oGANWBV9G56O7ve7IjMGQe+cb4/S70uyVCyRCGMcuKb1cZjAwAAAHMBmOBgIgh4cb/ABArJzYZ5rHX5j56r8wHgW+tF5AqARAs1BrjplFfvXEtrlwg5FDavOhapYMkKQc9sva+OX4DWma6THUhK+6jH9dqUztHZmcXznyic0LkrJvxuRu0Y+YRawXkh1EQJ76oQT8HclU0bhgMu2fJYChB5KELI+GQETFb5FVCxTkbj47NHH8zGmnyewi9m47nBn2VLDEvioMkwYf59gBEpGVmi3zYoy+KwQAYKq1EAO77YVILRMKDAACsVoWUpZROa0LwwigZa4nxCX26xwa8Sm77IpGra8HI2IXP39e3F5zNwAciUFmmBwOd5/HI1uuEVutcZmmR5lY5IC+IwqSx6/VfNHfuxS0j6gRjXjzKkmjiuGn/+z96BFUjPSJbyr4qszxP125dx5o/vq+/98h+aP49Dx4jRP9D4SkKsBfHvhC+qFSuA0C/qgQmQplVe1Fz5kcxEZBzpiyGP7np3N2lDOFdRuFwL7sNnjR4kfD9mh8nSvaJSOVS55nOpR/Fpcxke+3qu7/VMiW0ky8cXfnqC1uFojnKtG3StmAbYw15Bmi1XTl/DhlzjhDRFrK6nClseJuhrCosLNdXdS4BgYAFca0xsWf/279MO3f8OCp6RFqXHismJ70iY2Fzy+1Ffui9eOTIxdW57mC1YhiSIy+OQxEHQgAwInIECOg8stbFTjfGknji0MLnpwullTMAyJAVnU66ujC6e2/YbHzXZCYCcGJogTk/rjemd9cnp2+d/ZRlVYbKkTXOGgTUWucyL6Q0SsOw7wkhoSv1Qu2L1K4SOWSoiwwDdu33/2R6HVPkzFULDugMOsNoq65D1XTI2Patq+2bV8nZKoq26/jJiX0vBvXGdhoAE7m5KzcC5jsoiQ33HhLpPFdp36U9PqB1XYuj5p6Xv/2yfZ94sAA/qe61FWluMy8aJPUdx090b895mWwpWoqJGIa1sbA5unj2MygLImCeF8RDKiUgGaVMaQDAj0J0VmWpyiVZs+HTIQB5cbjz1Vez7pLKB0BUSHnzT38g8qI4ygbdG39832YD61TrhaMrNy956DkwBOCctrmqSn4Yu6uaeBj4ceww1mVpwSKgVWr5/JnaxFRQa1REYp1L2V6b/+j3/bVlJSURRrpWG9t16ORfB6P1Ius6awHAcTYYdJavnBOj4zwsKNpKnIIwcCJOhBgw7pyFqlQqDHad+kVzakdFJK8s6CpAxJDCWEA/9df6l698AYVBRHJOZenq1a+Cmth14hc8wO9GBsSgloS7m/W+qzd37z7+Vr9YSMKwK3scOSAUg171bc6Y2/NzC7duMmc2iggJCLSfl2FahgNXOgQgcvNnPkZAK2XFDrXA/LgmhBgdi7xo60bQ6Iwp+t18fU31+85oAOJBgHHgiZhx71EUNAAALbO4WO8VOQdEoni0NXHgxc6tm1ZlSkotM10Ugb8zaRxWioXBM69/HyrAuVy9fH7i8NHv3LxnWMBQrUQQi2RiavKFY5BKQGoAGI9zz+/M3/REUvb7yACsFWOtHS+dDEcaXhBYZVZv3uK+35rZZYt84YtP2tcvbQ4+5K/HsV8TQRIjIgEUeZ6upO63/8LrPC9SLaUppB/HqxfPIUFpckS26VkCVBY48CCoiof8MBg/9KJozV6/crl37SsctE0x7MtB5BC4KWS+3r72u38erCwqmREAQwzjxgHzdjw+xmq+L0QQijKMjSpLmRaXLynr8Z+9zRij4KG3ExEAC0MvSkJRK9JepW2jME7GJpqz+x5o6CKBiyTb3Q1vXipkCpwBorOm6KfZWqfs9Xjgo/+4zTfue3hhvbn/xG/saBlfHIVRLjtrmgGjIdk0rDcBAAGsNjpv66LrGWIceBB6ceKLhDHPs4Smqi1hWkrAqrYMASCIEoxrNLnn4KvH6mNjfrxVAAkBrHHdhRudm9eIHAB6fuTHvi9C9Ph2zikjJeWpkykHB4iILAyi5s69U0dfVlmWri4vffm5yno6l/3l+ZGZ3d9u0b5nPNSEVjLL2qsA4MfiW4uxH8e85ketJhEJ64sgrkVJ3JqE1ua3oNNKrt1hZBgjImBBaI21ZMdmDzKfaaWSqR0A4AeR6qdhfQQ9H5SGSu42nhoScuYhMABLjqTsrFHh9y3gsL5UFzlAZZPjsF7PkccDP6nxWDDuscAb239wdOceX8SBqFlHe4Gurd4c9Ffv+UBoijxvr994/58Ha3dULqvhfJHUWhPBO6NeFFsyGIqRQy92lpcQEazrFunt29f3Xhj3Tp5kwUPrXQmAhXLXi7tk+6syswSMIbhSmVyqLAtrD4hjEwLEkd8Y9aOGFwyM0Vg1tiDKFhb7czeikVHPf6ziwa8Bg7AR+A30EA6iNYp6QFgVOhIiQ+4x7lVcksj6kfMcOiLGPG/iyLHa1E6GXuDXgqhuZReGfQUAABy4IErE6Piun/8qGZ8MRBI9osscOmPKfjdbW8l7Hae1I+TcGzv4cn167zbrq5RMl8+c1nkOGwwSozUihbW6GBsXrfGRmd3tm1fX565unbd/pvBgAfZjMfPqyYUzn3bn5yYOH4X2aiCSx5dk8qJo9tTPr577x9k9v6g74V2/zuKvj4CMR0nSaMZFPzSlMlrxIBL1li7z0E+8wPcCHwAQkId+1JxqTu2xJiegoN4MhyQE8mMRhyNBJPKsx4ACkoNgrW4nPEIgQsRAJFWSoJQlAHKPx37IwW/O7mnu348+40HA/AABrCplb52UXbv4Z91bYwyrewJtkRfra6rfm/v4/f7yYnW1DwA4rvx6OPGL496OiCLHiIHnL99pR0lS9tYd8pLssuxQZ3F3pxP64+yhSpj8qG7FJ4FQwDywFoGZsrj5hw/CuM4A/dqDFh9RTEzuePPN/P1/pSIbdvOxVg7Weu2FRndf7LeYf+87IpalkanJMmktjxI/SaIk3iKUBQxopEqyE9zBKgnMGPcjEcSCB2H1dHwEn0HVCcULIz9OfJFY45qHjqyt3MrzHgzbh5AfCS6ieGz84E9/XZuY2samQrJGDQZrV8+vXjpPzhEB9zzXnPbHZ+LGCNtGq3olM9leHayvmLzqWAZRcxQ9rzN/vT4zg5yF9bovhF9LJg69wLiHPnvgmM/azW8PEWCRAMDMqyeVzFYvn69erCQZALYvzF4cx9g6/vb/hISR82H3LD1g8zoeiHjHQZpbAADGWL7e7S4uNnfuuvebCIAF3tTRF8b2zjqrAciLYl8kG1kV1ziwb/nmRQTg5DgwUrNC+HFch0QgUNQYmXn5DRb4nfXO+ur63v37gsBnns/9gJCsUVYrkxftm5e7t+bAOqMKJTOVZ+QsIOiiKHrdhbl/6awsF1mmcklEAMwn4mEtnAQxNlGR8ggojOKDx1641b5lo1iXBSGWzq7evMbj+uzrbwUjNfQfKMMI+YCyU5D9C7hBFWzXMmdmfeG9P+z/u9/4SbJZHqiy1BQSEIOkHjTqzT37d556a+7D35uyRARCsGRXb1zCIJh+8UTQrAci4Rtvakttz56bv/llP6dBMrr/1WOtHTvCJOHR9u4Bo+ouFFbfsXN8/wss8KxRpshMmZJV1YUvVf9pXRZW66WFm0UhKwIJARGRH8Z7Tr09uu/AZin11nDW5v1s9fJXC6c/LtO+0xoRwfNnDuwf3zED3oMl7V5omcn26tyf3teF3Lh1Cpnvz7z6ZmPnzGb2nHEeNZr3ftRvjvOs3Vb9UBO6KrbyYxG8elLJDAA2JRnuEeYKW4h0GA1X5GERBgfYKwcLNy9rzpi2oC0WkhUZaA1wLzmGeODHwUg8OvK1EapMy9KVc9apikIYkFcPm4ebY+Ltn1MSI4AfCwSw1sQi3rVnxoEh5lmnTFZ0F26uXb9E1lqtVZ6pLCOtYKN/BzcOrIUsu/3JH8FjJpeOiAAt8kDU4nD0QCOMXn/H2/DfEFEkwpuZ1a+9ef0P/w4EyJCsKbPB+tVLoj4y9errnv8Q2kQe0+e33LIhx4CIEIGhloWRucmyYeKNSMl0cGdh/sP3WKO2+9SvxZgXNZq18SkxNm61dkohojNWZeny+S/ac9d3HD8xfeSlsNmo7hxjgCtlZ9Dpdgd96K3d6C/ofTPTb7wTR1vdA0bOqSzTMjNFjoCM8yBOfJF4QaQL2V280Z2/5owFRABnSl10uunCwvzVL3s3L9usAAsbDBAkAgahI25hW5fWILD+oH9nec2r18gYRZkjEqPjjYmppFnn/OsOsJZZdQHi5iuqlHMf/l52Vk0uEQAQvTgWUzuaM3vE2Og243xP/G6UJ4JHpJEqMRatCS2zTUmG+4W5wtdEGratqBGg5LAqTKNK9CL6qAC7a7DaAJ7Ao4uqEaDIizLN8qyodoSHbqIRJG//XLTGSgBTSCXTotte/uqMKRUBIA4vB3PaVNd5Gq3wnvz0RpUbIqPm6nqalLqWbHCGh9evqNrE0TdONmdnAyHo3ggqY36tIcYn49GWM8YqhQDOmixtt+/cCu/sjCdagYi9b1xfSkA8irxY+GhVmQM5ACLGHIIDJIBC5lpmg+WF+Q//vex03FpSeGde+fnb2BC18emJw0ez9rJVwwS7M6YYdHme3f74PbRq+tjrQaPBPc+GXuPIq2urq5SlTmXUydbCYCRN4/GHlG4MJ0fZ+trSubO2UoB4tzrBGZMP+nLQt1oh40DojF798suVy19pk5V5FhjfDlvrIwJYpa5+9Dtv8chLP/1JLKIt3xUAwIFtjDZfePNNK/PB7Rvta5eNg8kXj7d279OqKNLuvbJa4Wv7U5VSR0D3MNWi5uiu4294InosT/cJ3o3ypLDdPPCmJAPA14S5wjdFGh4k1RXulW0CqgW1I5Mv3rr6vgeAgLk1H5V/bhbuV+Y/obetREgQC12b9OM1l/cqXtKAWVmk+YpaOPupLQsA0DLVUupCDjkjG3zeagQGAOh8ooAAHObcdjHy0QVkBq0mAjAcckN964Io5qIRjE+J3bv98dY35kfAsDaxY/aNn8794d1i0DNKAZJzpn3zajdrz7x6bM+hQ+iNEdzPv419/MnJ6dmd6Wcf6KVbVR0hoVUeIwYAkMveVx9+ZOcvkcx0kXOONSlVkccj9aDWSMYnRauV0h2rDDoGgIwhGa1zmD/9CfjBxL4XgDNbFkXat1UPW0BH4AjdAws47wLJOZ1nFdGtKm0CpIr6prKByTI27KgPyMDpMtWrkCEyikUia8zXSEUGzgGAzlOT5xG/1L017RGFSc1LtkogubJAVaAqEWw81poIjwNAIGqyt8Y9f+XyV1+fK+DI7H6dSyDni9iPE8jlSHt8bv49AwjkHCeMI0+I4NHtL+/DU7qt+rvg2xA57hXmCg8UaXiIVFfYkG0EomLlTuf8xYh5BhUBpa4MFrBeLwZs2W8wbxsd9C1Tuw/tunzrbMXfJwCTZdc//F3FJdK5REREpKpXFkHAXCTIiyLt/DJl1liWxNFIXPfsuA4i7d/QKWlp88Jwj3nDRpBVOY8Xi72iJd75pT+942Hd/ZGxoFYb23OInJv78Pc06FmlnLUm7TOV5c2Lae1U2PrbIPGZt5nmIYpjDzAQwrOm2usACByV7WadFfAoXVvJ7pxx69KrriEhU8g1kqku6sz3G9O7Zl//ycKf/lTeyZTJTaCAAJEZY0hmt//8kStLleb9O7etSXVeQNXpD+gR1UgbNBoGyHGzkRUhAyUzJQc3Pvqdc8YaNWxpC0h3Wc0YGLvnhdcW1xfyNeekrNJjHBzm6a0//ttgZGb61BtbCzA8aCOVwxQ0TBw+ev9UEf1A53rt/GWw3ekXX7YEYBHOXqwofUO6a83z48e7Q/wp3Vb9HfFkmFjfFGl4uFRXqB4JAjhH5aDvcaaHDSshsl4zjWs38k56OjxxiugRZyQCqjK9eeEfyRUbBC5QMlUyrWwmtlED4Ah98LiIJpLazH7i++NB70Tn0nynyOsH9+/auyv0WOAAie1PM/jik7W5y1RRRmiYAuFRHI+06r/62/r0TkhqW1h/yFhQr43OHlAvpTc/ft86BQwZAqd8/TrrL1/c+bKYPvxK2KijN0IQbhCdOb90meUFbDTSRSAq0ouf/tOucMd63gkkaSQkBkAWrGzfWf7sg70/+x+8sdGgXm/tebG5vGv9xrnbwae50aaqx0a0Wqk0vfXph0BolHRWI6usCgQCco9YYSJnZQ65tkO2HDkHRT+TaysLZz8u87RMB4SMhj0wYdPfJSLnOJ0/P3vs6M00k5nEjWNDS2ky2S5V+VEWT0xsfZ3316R0Ew9y01Bn6eKVM/2lq1am6fIi8+NBwb28h3kGDBiwetLad/TnPAofN1P07MjtJjzYiIw/8ck9UKor3CPbWPUN79yKs/bKMM9AUCvtDjHmHzq+eun8I40cBCzLXq0YQ1wxUOq75BHCoaoAjuQsGONrzvcFjejoMdPyCSK/5jVeOCic9eLYEmhiCgkZKoacI0O0wysx0Y9jPxZsZHT3W++E0zthIwC+1cQYD+uN+sSO0YnZXveOKXOjSmt8M3Asd/OfnWPlRxOHfsHrP/eEVzEBTZ7ZmVm6fBaMA145yVRKGRU471JOzAIx4FV9Hjkn81KutfVX54K3f8mYF8YjwbF6SCk/c+OG16HSWR0iEiKzSllQ1YDD4oaNNq8ml/n6Wi8I4C4l7f4FJlLr3cXPPrVaA0MgIGNVr3/tg3+xurRai9b44M5iZfuQYzyIAlELajHjzHckoxj7nSip2bIo08Gw3xgiMNJlVpbZjtmTojXxoLcGeJysxwZYYtY6ZPI8U1nfAneABpGzamfwJGlFcT2Kkme/Z90j4QHAlXd/+z2Hxe+TbYL+6sLihdO+EEWvV/VClWFQtEaajZFo4qSSg0eNBwCIMi/ff+8aLZs8AyBADEQtiEUVfwYAILSEAeNSCJNJshFAsfnzagByZWMsYLbfs71+RFjAsCCPGRrbe6i2c6fTSq6vyvW17cyKrLVOj73wAtz0Vq9dhGrjIiNjdLY+f5Z1r/95/KiIpw9WXXVdoZe+PDPQJXC2SfxmxLRjCGTRDYsSNigRxtE6+INmzcxfdVEEAKiJpnTwlm5eX8tujQ9JCxucF0QEws0WvxtBO7Z29XLR6yB7QEUfAjprB2vtTA3KQlVGNONMyV6Q1MNGEzZu5bZa+aIWhAl605MvHhqdGUO+GWlAU5TXP3vPOaulhI1+sr5IxNjkyK69ojW+jfXcFjyfje17efXWQtZpEzIOwLFi8FTy6hSU2+hV9uOA92//2/8yMrv3BwyLE7kwrk8dOp631wXjtiyt1WXgVISsLuJ6U7Rajx4FADO5/s6v/Hd/a/OcgwWCUDT2vPW2aLXoIUf7FoNhaeCrc8vIB3x4Q4IfC2d1vr6Wd9qPNxags4YFQXPX7ryzrgpJZcnD0I9HPSFcmHSWVge9vLqMy+RFSRn6iHrYzf7eq7Crhu9AyBwLAD2RsFrdHx9Zay+F6frd6kIonKlD/eT4Pmd1aE2pZKpl6owdTqj6i4DI8TDmPJl+6cQ383N3PwFR2Oul575qCGHygSlyztnYvr07jr3JQt9p25m/sX7jMiC09h9u7tjvVb3jkoh5d6+e1IU8kvzd1ff/v/7igpIZEFbSu/etd7a2nx8TxIOYQgGBIGBEtroHAu5yyVUIBcEjCid+LBj6wA8Li38PvJPqTuex3QeRYX9pPogb7euXsEbRRML8x2itTYnwx8fN7OERIp0NEFg8Oj0yuzceaz0+LQ6xKMzoaJhnytlHBGi3OSKgs25t7sril392jPtxvPOl18b2HkJ+Lw8BWanKztri6X9W0rnNdlV3c+EE6EJC329OJGPjky147QTFW/frQafN7QufrV2+4FLJgzAQiRcnjHMgAnLgR3vf/EVzx0xQeyiRg4jCVj4yvdPILF+9s3zhbH1iateJt8LRES+IrNHx2Nj4oSMAECS14G736fvG8COBY3Dw7d9c+uC/0cIyOhCjE4d+9ZukNeGLJyjAQEBeFEWNqfq0tGVmi1zJDGh4jAdJEgjwY/GXoYLxv/znf3jjf/6HB4bF7+OdPOW4ubPWqsJZa4rCOUNkgqTmxzXOH4OVWuSFlpleW3NnvrzK2JG3TtXHWp74vq+BfQjQGVP2+suXv1i9cnHi0AtTR14OG417otAAgLY3KP74+ytXP18fdJ3jmxJVVb0Bki/CepgcHNkTvnHKH29xEZHYOpqKRW+9Mz935d//uzNGjI7vPP56MjldUVCJqHIvA7FVYfAmqmonUxaM8SCp8cfvX6ELKdfX5j76V6f53rfeSSYmgye/r1DnucqyKu+1/OVnKh04R+TQOd4Y9Wbf+nU8Ns7j+o+F8LwFsOv6WmYPk96KdyJaE4d+9ZtnMAT3QKDMi7wAABLxoyjy3zPQGa2y1GrF/SBIal/TVAisXFmZ+/CfVi9dyMsSkN3lSzgIhAhEmOxs7T36MzE26YuEtnc2aZldefe3sr0KAAd++TeiNRnEAvm2WFBPAzrPtJQE5MfJU5Der72XNBUrqyqIAvJFLUhi7xuc/B8pPHi4efwM8k62AxJx+Kxo3a+BmOdFzZF7X/naN0AU6lrNCQHaELmN5rbkC5G0xvf87JfJ+IQvEi9KHkt3fD8MhG06XH6c+N+X/Pix8GMRjz2xCNmzBuy6/gO/8H3az8+xATRlkXc6tz79IF9fBaKNKwspqMW73/yrpDX57R7E9xDLeL5hfhA8VIDh2auc+o8BNEWuZaZkuhk8RxxWZX9viutx8eN1uH7s2IqJ9fwZ/BAgL4q8KIrHtpU8e3bwI3W4vh88PV24rbutnuM5Homdr7whWhMHh2mh50f/XVTmyZV3f5u1V/VDmMXfGluZ0M/xHNvHc4frgXjazsVTbCv7HP+h8Jcqt9/9YHqqzsVzE/o5nuOheCLW71N1Lp6b0M/xHA/Gk7J+n6pz8dyEfo7neCieiPX7VJ2L5yb0czzHQ/Hsh9afm9DP8Rw/Yvz/Tf8RxajsxTYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<PIL.Image.Image image mode=RGB size=320x64>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from captcha.image import ImageCaptcha\n",
    "generator = ImageCaptcha(width=320, height=64, \n",
    "        fonts=[str(Path(torchkeras.__file__).parent/'assets'/'SimHei.ttf') ],\n",
    "        font_sizes=[40,45])\n",
    "generator.generate_image('中国人民很行')\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-18T11:19:45.704071Z",
     "start_time": "2019-06-18T11:19:45.700019Z"
    }
   },
   "outputs": [],
   "source": [
    "class CaptchaDataset(Dataset):\n",
    "    def __init__(self, characters, length, \n",
    "                 width, height, input_length, label_length):\n",
    "        super(CaptchaDataset, self).__init__()\n",
    "        self.characters = characters\n",
    "        self.length = length\n",
    "        self.width = width\n",
    "        self.height = height\n",
    "        self.input_length = input_length\n",
    "        self.label_length = label_length\n",
    "        self.n_class = len(characters)\n",
    "        self.generator = ImageCaptcha(width=width, height=height)\n",
    "\n",
    "    def __len__(self):\n",
    "        return self.length\n",
    "    \n",
    "    def __getitem__(self, index):\n",
    "        random_str = ''.join([random.choice(self.characters[1:]) for j in range(self.label_length)])\n",
    "        image = to_tensor(self.generator.generate_image(random_str))\n",
    "        target = torch.tensor([self.characters.find(x) for x in random_str], dtype=torch.long)\n",
    "        input_length = torch.full(size=(1, ), fill_value=self.input_length, dtype=torch.long)\n",
    "        target_length = torch.full(size=(1, ), fill_value=self.label_length, dtype=torch.long)\n",
    "        return image, target, input_length, target_length\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-18T11:19:45.737300Z",
     "start_time": "2019-06-18T11:19:45.735033Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SUKD tensor([12]) tensor([4])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMAAAABACAIAAADDDu+IAAAm4ElEQVR4nO196ZMd13XfObf3fv22mTf7YAYY7CsJEuJOKhQZW5JJybZcNq0oXqpSlXxQqvJVf4G+5kPKVfFSlbLlRFYWh2ZsKtpIkRQJUuAigsCA2Aezb2/pfr3d7r4nH7rfmwEwy5sNtqvyw6vCm5le7u177ll+59zbGCYxPFg88f3vnH/luw/4pv8fewRsC1Aj9CacxdduXnh57Nxovqeombt7p/T6/+mT1wFgtNDz7Ye+sru3QIAZr/4/r52/Wp+MEyqpuS+PPvzYwGFDVmkXb/NPEo3QS7/s+qhtCtZuQTq6E/biazcv7PptBFGNO385/oYduQjw8ti5TU8hoHro3nEXbrmztbiRwKaaEp3IuxXOXazeubg0caM+t+jbkUgAcFe6kAix7DuzzdqiZ0fiQavtDZCN3a9en3AW25L0wCC3v7XlppPR7RxuHDZCvxH4l2p3Zrz5KIHfP/X0YK685lyJRNII3UboeTEHgCXf/tGdj3Ml+LWR0w+Zx2RJ3kyXEBMxAuyW0LSBAILEzdr8j25/eqCr9/GBw926VVRNxF2+0VaxWq+/dvPCtx/6ygNuwIoAvTx2btftVxDzBa/+6u1PrlcXwmbgxEJX2PX63GipTwmlwt0DkA7SeHX69dsf10OPATZC146aVkM+pg4f3x/rEgJsJEKMUItUSUgxJbvS/lXAWCSLvn2zufBx7faHi7e+efTpQ8X+B28yUkRJHCQcAMKEv3bzQj10/YR/4/ATfhIW4YE2KROgomaOQk8qv7v1UGIhar73o1ufvjP5WTX0IBFAIiLlx0tX54LGK4eeyqvm6vkriKpB85a9cHFpohH6iAyBFGQ9sqWTIYO8sfQAABBD0pEkgF0WoASSKtZqrLaQLDb8mIvo1esf/JvTLz5gAWpGoSDRCL0w5tdr04Ko1yofLg28PT2uSuzC4rV9+W6FyUXVlJn0YJq0ooF28VkIIj+Kgzh6f/bOh3OTjbCZiAQQECEQcVBdlML40/IdLpKybhU105Q1BEiAZprVX85di5MYARGAiBjDg92DRyv7ZLaJ/WrGfiPylxInod13mhmgA/a78K4MMoqoGfhzzfq8U6+oeVPTdv129yAWST10G6G/FNgfzt/uyxV+MT3eCL1TlX2fVievVGdkJrlx+NHc7Wm79uujDz3cc6CsWwX1QQi3vPkhWwcBzbmN96cnZvy5uWCJJ4IhpqOKggDACb3/cf39Ls06XOp7Yd+Z/lyppOWIqMl9N/T9mDNgAEAAKlMsxcyrpioptL4GQgCX+E+XL87xukCx646JAMpD4Sl46vv0LhAAUpKIK9Mzpysju32re4EACYkr1ZnXb3881VwWCHbgxiKJQMxP2yggiHkkEgBY8h0/5n/z+S/en/38Gwef3p/vzeuGKu/JELexJ1evUu12OPnzxfdmG25EEQIB3eVtOmHQ4P68W59pLl9evvP1g4891n/EUvTszwSQejsEAAxJos19VXQTvylzR7iCRIft7Dz6RUCFGzI35EgD4AAoM3ZsaFDsgba7B4KoFjRvZ5bdQ0SBhAQAyOMYiBCgNTvBj0I/Cr0obLg/eWbkyJMHDvfI5Rzk9q55GwnQttmFRtz8H9H/ngzQ5wIQEfCuqIgAgSRCIagaOF4c/Per53OSeaJrWAAIAAJEyAIpwShWOhokRqjEkSQobmmqjU+7i/eCTeIGIppzG+/NXAuTCBEJwdB1Q9f22n6lln3WrX0wey1KYgQAIiSKScgopQe0+4mUBqBkh2EYLyxW5xq9U7+nvGxJmzqP2wdb7w/bZhcIqMlDeaHfiXxCTNtOQJSqFCJAyD5EjFgQx0ue83eXP7zVWLC5lykbRAAgJGJESISbPwEpRt2W5XDdHq3Ru455r0boz7jVWW/Zi0MAsFRDN5W8qm964o6BgoQT+i73/IinuiYSCQEllAC1Z0pbiAAICaFJ4XItqE6zRa8e7WWyYe3HvRNe0eMRd6RwGRVupHpHICCT8qqZV4ycoq3qdupOoBeFN4OZ733+5nSz5kYBZtOKiBAIkDpidVCgyg0UrH3ljbEl3qseuu9MjrtxCACAaCn6i/2nC6qx3vGN0Es/HTR8cyAAo8yox5Skj0NCliqk9EdCohWWgyQhQahdn61eX1yMKNk7tmpdE7Y9XhEBvIi/d2ei4QcgCAAIwVL1gmEdLQ4fLfa+NzU+69ca3OMiQSQgBAAB1Ii8yE6u23NIDACBgIAkxgqKUZR0Q1I3FQkC9JVIYKfqukPeCwEcHsw2a5Netcl9ACAGlm7kdbOgrH3WloxjJyCkhAlCQAAJWQxCQgYEOVkragYguklQD71WzxEIECBKkpnQueM3loLmUE5h2JFu3ipWBOgej2e7vCK6kW/zmsN9IgAEIjJV9RtjX3ikd0wGdrZn9M3JS1eqU0u8aXPfCf3MxhE43BckGEqsZdllgH257nN9Y7q0iR/Ek7AZN524ySFCwA3itXYfO+e9nNh/a2q8FrkEwACJKDGR1hmO3aWGCajJuRNxO44AUiaEVEkuamZFMXvVwmPDR7vN/N/cfCeqi2YQAmLb2iNiEicfL0yc6RoZMEtSx1NrS8gE6P5Js1VeMZU/RBCQRMhjjAAJABGwLBvDua5RqyIzyYvD39StRnh6xqv+zfXz043FZhjASurhrnBLRqmg6gXFMGRt4xjeQ/ei/ekkv5NAgsCygzdU3B32y4vjOde+yRft0MPW/SSP8L5QL4G4KmpTwfKffPbjKccuaPoXh7+4XrM7DlAwFuKzhSUJpNSFBEAGOJLv+e0DX9hf6C1oJiD8gfov/uzyD+dF7PKQUCZCAEACHkXca3qBl4hEYR0wsVsHg3U8nqJmpp9OrrLicdtLde5GUiIw83SYwlA1maqkasWQ1f5c6WjX4NnesX99+NlBvdtQtBUnsD3kq6OLTkCwWFj0VZ/Rbpp7BIjj+KOZGafJszAACAkkAfe49QhABLeDqT+Z/a8LjYYTeiqTSJAMazDCWwlQSGbsVE8vERFQRIkAwRhWtNxoqXes1N9t5Lt063Bh8I8PPXeqZAzkTEVWBIgoC0aF2AupWYVMEe8kk3qP/BEAy/Q7AhAISa+YTFLuOauoGMe7Bn/36JMHrd5uw5IliVK/Of0ztU7vwIEmAEVSn1KeHlVGgW1V9DYGNuMAk9DjPglCQALIyXreMgqqwVapS4f7U82lz2eX6re0auzITGqG4XsTVxzu39OBLQUoCGipqqUqhiLHQhCQALJUXTVVS9FbjwlLqnmq+8DvHHvpSO/+WERpmBaTIEAn4XYU2NwX0Ck9tiVkAvTy2LnRQs+3H/7K9jKp6YNAgBf2nW6GgRP6ggiABEBOVayGmmcaw7smLSIWNPNs34E/OvP8se6hLGxfZcJwVfM270aiytyQIgMFo9Vh7Y4hQMyxWgRR+iMB6LLyVOVoRc+32S03Cqbd6l999tbrVy/5IUpCA0Q7Dq57C5/Xpmfduh/z1dfcxnRtkSEgo9RtFL7Uf8pUtHYnGUqAagLqzWotacWgMjJEDOPoFzNXFnx7jyhPBqlHme/59kPblB5YJX+9ZmF8eXrOqwnK6GSFpEcrA92ygffpEplJ3YZ1sNQ/3FfMFxlqnK06CJHlVT2v6DlV31ggECCMow8Xb92xlwSlwcquGbIIkjiBqJ3eJ7IktSDrBdlgiGESLfiN6/W5vxx/66OlWzNurRkFmQEWNO/X//rq229NXw6TaHWDtjFdGRACykyWkJXlXFE2LfleFurHE78Kmc9lF/VIYojAGKAbB07kO9wn2lXr3sJKNn7bl2h73Ihgh14jajo8IBIMWUJgKnJOVS1VXTM4IgBNVl4YODmufRiOK16Uuk6p9w0Vo/DM8HFD1jabPOgn3Oa+HfmJSCTM8mg7R5DwZhi4gS+SJDWohqIWFb2g6BJjkUhqgfP2zPjrtz+pBk2b+6kFxqxrVA3cmMSsW/NjXtastFFbDVAy3QMoMyntmMN9J/LdKMgpd8nQy2Pn7Ju2MdJYvENuLfW9EGBP5KaN3eEGMo9bzXIumQNNqRezCYtsSGqfWf5987f6tEqrr0hADKGgGgXFsJRO0gUEeI+/uFM9hABhEn20cOOOvUCC0lbJTHq4/9Bgvtvh/qSz9ObUpVevfzDlLNeCphACAbHVBwIgBAF4fyJvSwEKZg8SkRABBYil0P759LgX89XXTc3Iv3/4K/+q9Fv7jYEW6UMA6PDA4b4d+XsYxu8SKCdrZUXPa7ovIhAAgEEc+LHvRZ4uK+uNaBGKZegqQ5lBPQFCSBOEqIgtFPzh3Soaaae8BxfJgteYdetV30lIIKAmyaZpCo3NJY7rLPzDjY9uOwt26PE4ZngXYU5ADLCg6Iam66outtsUAnLCoMkDj4fpsyAim3tO7DUjv9corp4zRc0sgZnIellM60ngQzP9vRuHb06NHy4PlrXdz6quK0DbyKQSgCGrjw0eu1ifWvSbAogBhEn03p2PDuQqKuqWvlHyKPVdMv2PBEitDNrmQAJMA7DsUoBAbAcaCAFiEV+pzowvT6fSQ0ASk46VBm+rzQvX3/AWnQb3bO4TAKYpP2ydCQAAedU4UOx9euQ4ysyQN2fS1wQBVIPmu5NX/Dha1VlCWDc7kVP0MX7imjLjx81MA0WBE3lO5HcU024RawvQtsl4Q9HzmllQcwwxAcGAeXF4zXF+9PnF7rNdGwtQyv1RVqsACRKXo07SqFxEqZ/Ik3ilRjZNh2wXCYll35l1q4t+I0oSlrKiAL7g0ax/Y2kmDEJJtIoGWmXYAkiXlJJqkR6WrfxvHDxzzNxfVE0ZpZ1oQ5FVbGRdYyCpiczW4sIJwFL000e19y+q8812//eQCVqjETtdoUFAwNJpCQiCWIOHk9jowAZjOw+afgSu4UDcf04kkvHq1JXqZCIEtp0P3L73mFZnTzqLn8zfiNIqfUwLu+jK/NSNhRnkAlPOIe1ky9GNEYQsPTRw8A9PfenEKWVfsWso111QTFPeZtUHApQN65nhY7qsZKwPYl7NFdXenGKsKRmaDroOut4uo0n9+r2SobU10E54RQQJQUJg1Gp3guRwvxF5Lg+stUog3Ch0It/mvqCMO0SAgmJaSsFS1s14t2/Ik8iOPDvyeRK35jpCp1n8ta8ZC9GI/aoIgjiSgAEAAvKYhwlfCW3aExwFIBY101SNUqnnYP/I6Ur/0/LhnZsMBMyrel7VLUVfgAYAIrCymX98eL+hbBScYvrsV4LCvVo+srYAvTx27nvjb7009mhR7TRYaIEsVa/I3QXVXA6y/B8Q1Hz31WsXNFkZtLpKqmlIavsEBPCi8J2pK8uBA6tKEgqq+cLQ8bxidDJ3kBgSu/sZyQjStkeQAIkUFDKlSppWZnFGebaPRALAnC7vz1e+ceSZilksarkuZhq7FKC0aK0sqETEvJzLqYqp3Evut1skkGImCFcXgnaqgbbq+65hwoqaWVTNl8Ye/T83P2zwrRW1EEBO1p4ePtillhjILYNETuhfaUz/x0///odTH1b9RnzXwjx0o8COfIf7op2aR7RU3VKMvNJB0RYhI2SrajszJbSDAgYEIFIEmRklsSIwuIrSpUSWuWooVnk4N/oHx55/uHvkRHFgSC8YbHekJ83GuzzyeNxqBgLEuJF5RgUllFFBud2XDrGNKsK1faAG9743/lY9dLfhA+VUtd8q7rP686qepWoACUQtaE7ZS2/f+tWlxVuJuDeIYGltLwAAIAIRJB0Pf9vvSbGiJ3YGBpFMHqTNX7lqa1KjABYZTBT6xwYGH/7msWcPFvt3fSEEAdR89/zUhB8H2fygxOVek7te5K/fcnlQVACk1lPATdk42K7vu/s+EACUNfPLY8dmvNlExG4UAgAQMcQkShY9b6bZqPpOn7VS4rQq/k1dCwRAWeD92Y/7EVNiR74d+X7MW8eTLil5RSsomsKkbcQgRMLhXpM33dhDFG1pbPGihAAMWFHXegrmE8XRx4aP92lGfg/WPxCRHXl1XnWjsN27ul/9YOrjsUJ3bi37jgAF2RSSllOMasBZSiZx3+F+k/v59asoYVvjvvYs32Fu1VLVw+W+3z30+KDWpcnpcpyUaGORoLenLr9X/WwinK5CNT1+ld0BAEAABsAIN501CEBCTNiLHy3cikVW60kAEuKp7sFDpR6ZbSd+bnD/Rn3uH25/5GcuMxbV3H6rcrQ01GsW0+o2IiopXb8z8sKLg2OH9VxFUaW9KfkjSBJMCEQrQwIIJLFog1NMVf3C8JAiSxHFAkgA1EPv51OXvTjceEZuK0l3H3aeW0XEopY7Vhl6fPiwgkxAVhWPiEEczQfNH1w//5/tv25EbusMumspFwESStCJBsKYhMt9j3t+HKXHCyRNlvOqWlBTDbQ1NEJvwl78i8s/m25WHe4jICPs0nKvHH3mj089/0T/YYlJgkhikoqmHbC+XPeerr1K68gIWukhwJJRenTwnKHo682NBOKE4kRElNZQpwvueNDkwQa2fXvjvnbPd17GKyEzDEnJJ5iLEyeShJb5EoBuHNpuIF/vmTpe7bEqMkp25Dvci0XSLgDKKZqh6paqd7B7weqHC2kCg1BQuoRxW3jt1gWb+w730yq+vGKMGb0nu/flNP1afUaXZAcoFkJAMpA3XB5oRm4vi7YI0/JyBCBgiJpWzGl5Y52KbABwQv9nExc9HiKAfLefsDE6Gfd7wrQ90boAAEAKkws9SXkAKnpRShcmt3wJxnW3Jn5x+7of8zCJfjl3fcpZFivFZKQryjPDR7qMXIe+MK0mfTJ9t82qICfyz/bsd3mgMAkB04V8JwdH26KcRmWI4PDm+NLtzQ3tDoCIBdWwVMNStHb01zLUa4OAbB4ITs0oVFBGYNk68d3gEu8P0/ZKgAhAZtLjxiP/YfT3HuofK2gGtnMLBERQD5rT/vwkn5mPl1IXLxECMyeacqqWV7W8qnfiRwNAFsBnmWsEYqkftdVmI4CK0sWFCYVJWY6LwFKNK/UpU9VWNyUhYXOvEQVetJFd2DFIZvLDvQcEEUudd0Itidn6woCABc1QDaOs5doFegJ2oaxjzTBt7zQQ6KAPSH1HzQPfOvjckXJvQdOxlSNPw8qZcPon/A03CgAgI8xWIAg7qqGjLGoDgMySqSjn1XxeLeqSslUl5MXhrFubcet26GOLOhRAj/YeTBekprmLLNOPyIh2wHd3AoxE8vHCbcbaI0VA0sa3TDR8aHiEr9qZjZEEKOFaBdpbwms3LzS4N+EsnqmM1kMXNhWgnS+Qy8nqgFl86cAXyppFLRIXAROimiPcCXPJtXmS3O3rICHrhH0XIBweOFHgRby1GIgYsrFiz/58lx9H9a20nCfJktf8ycSnU83ldhaCgCxFy6tmQTUyRihz5kAAxojbM5QdgojS8Ntd5f8mbJOpJRi8IU0wxtr8SEE1upVuq1Xbub1hrYduKjclLffp0kRJy0EnGijdv2jbMkQAedUcsLoqhXJOXUnfIEks1Kfn3VtzVUGEd0kQopCyFYYbX5xgwWucn7kaJnEWggFIkhRTshw4f/7ZjztnVBEgjpMLU5OXFmYb3BOttC4wQJTarUNiq0rGSCB1sPHDDkHYUrIE6cLljdw7AnC4j8uimal2IASJ4am+3tTLbITe9ijiVG5G8z1F1WwTRZsL0LNDx1+7dWHCWZywt78FX7ee//r+Ryv5Qk7WUocXAWIh5kN70l9e9u1YJC0HCAhIErghWw/Qypk3edDkfkZXIgACF8l84Pz5Zz/dYjUBRjyKms3A85Mkq4AmEIai5S09rxkMWSs3mSVTGXRQLbAbEO16I4RYNSJDizfiA0nmQnaaciSg9Tw5hr9cvtKI7QlnacJZfO3WhWeHjm+jJfcTRZsQGNkq1QPnXrt1AQCeGzpRDM2SlttqnF9QjROl4Vf2P/X9z9+Z9WIeJwhISBySi0sTBJSIlUUnpqzmNCWvaqwzA4Ht/1vH3qrNFjWTtsakkyZLXZJUkuWFmMUgCECX1YqR/7WhhwZzJQDQJLmkqEVFtSWFixgJpR3XPXYAXFULBAwku+fYBt5MA+r1ZNmJlkgkbQVRj7yJaG7CWby2PE9ALx84t41BXLOae3MGrC1DDe69NX0ZAJ4bOgEOFNUtSBIDLKu5M937pgcO/t3NCxwEgYSIYRLxmAMCtWhDItIk+bGBwwNWiXWQwSEgsTrVCSATjhUHTUX5o1PPd86JEYAsSyfGDl2M5ucWglroCkhkJj03cOJkabiomAggM+lY99CR6uCMW6N03dJexvAA0IyCJvebaTFhVugi984tYc/B9U5BwDfxjQgJKI13QRCVlZwUaB/O3/yN0XNbGrh7cP9ZHVGoaQV4I/RWixG0JAmgI2GSGOs2rANGaUgveEkjFiKtPiTEbJK1C1hIYiB1WIxBWTlOpqpkYD2adbjQ/41jj6f753XSwRSyopTyhZePPb6Y+JeW79QDz5SNomIV1ZyEEgGpkmKphqUaiiRDxHd/I7T7gAAfz99sR3oIQFAjrAHUAfJrHl8P3YeDJz4If5xucJyQAASD6V8defR098i2RWc9bIGDv0eMXHBfnX4HAHKQ00B/duh4KkywvjwRYE9xoKfQ+3mzvvJLBBQIQJRG+UihiD6cmz7ZPVoxLRnWpTxSb9GJfKflLQIAQ+jLFf/lgdP9uZIuq+ucui5MWevPlb957Ok//dSTYLksF8pqUZXaq8rJkNWCahRUo8mDVeVaewUCONs39tHCrRaDhrHanNn3fqSebh+TOsUNnrmnLg9++PllCDQhIgCSEAGwW8+P5HtGCj273tyOBGg1e51+EOC2N6MO+4Irx6XRflZ+e3p8NdfZVk7ZiapZ1nMykyd9Z8pvIjLIluohABATbZqLIXpJsBwv1iNb0ACs7wYRQJ17b0xf8uIgrZMRQCixnK7mNG3j/Rg2gKXo+6zK7x99+gfjbx8rDR8qd7czsgSgSeqjvWOXlqfm3AbsLILvpHTLUvRU57V/k1BUDfyb9izAyi/bNgEBZ92aQObykLWKWhmgBNIeMX6bC9A6BfZoyOqAu+99ZfbZfG6UVbrHzqXMUop2l9r44tAJN+Lj1ZnZZoMnMVtZyICUpk8pC1UJRQ1nq7hQhUof9Kw3yx3uz7jVWbfu8KAtZQkTkSbEBkxtByioxpHy4CvHnq37TsXIKWzlcoasWqphaboksZBiO/YasVcRXGPr1QeujU6WLTRCrxa6E/ZijTeTrIiWSlDunzn4qTM7jWF7hjw3dCL9EsXxvGv/r2vnGYKUVdgRYja5tvs8NsImArTBbjdSovqe5C1414fnimWjolkHS/05WUvPSs3c6kv9dOqzWd+esJf8xBeQUhLIAIEoYYQEUjuhReRyeo3/8Agf6tXX1roI4EXhm1OX7cBN/SBEIIRIQ1+Lkq2N5n0XRyxquRM9IwCgrLO9cFpJcsOe+6R6e8Tq1llnOzkCwKqn2uDehL34rePPldZZsfXzqUt3nCVBorVzKBWo/NWBJ8aKfe2tHdoOAwI0Am9icTFO4vZOo9h2n/bG399cA61XZEQALIbQC3945cJn1sTZwf0v7XsEZY1alm71wTb3vrr/kdfnLr/fmIySqJ0ijkmYiioxCQD8iCfZqy2wGYb29a5xNhuZyprLuxBg1q3faSzXQk8QsTQpjxSrgdsLiRxu73G0ITMpv055YZrLQMBIJL7vh74fxFFeQQC6xx1ZD/XQ/d74WxPOIgAUVfPt6fE1NRACnu0bK+nW1dqMlBWXY1nLjRQqZyqj7D4GKkiiOa9+sXlnPmwIEIAsJc4TRlymON1gaLedts0FaIOtyhDA5eFy0PSi+Fhu0O0PS60V4PegqOYkJo8aXT2oz6KTrmkiIAklQzLO9R2ccZev1WeIsjyBFhmJw3527cq1/FJBM+/3ZhBwxq15cUAkBAmRUQBISVLzPrvFTon1ve+dAAEWfXspyN63EsTxnGtfrc7OavX0dvfbbmilDtIkQPoMU5VT0nIvjT26nvopaTk3Dt6cv4TIWJaDIzvyJ7lfifyyahZbRyYkGpG36Dden/r4Yu0OF3ErEYQAYCkmFSRrC5npLUBOu7eeH7fxTgDt5kRxdK05fas2WOWOpepFzSwq1mrNQSSC0I0DT4S+tMoYq5I8ZPadKI/qElv27SW/mW6tCYR25PXFhUd6DwxYXffc10/iRuiaqn61NotAErKUJUkoLidGZarySf7GBFR3pYDhvi7jvFdvhC4BJJQQg9vOIk1fKmq59HZtd6SNeujesZf+avrn6Y+rJWZj+gMBrtRnGk1/1aJSdLj/gxsfnJeUryt6v6IV1GxLglv2wg9uvXttcbYZeAmJFo9PkgIFS/3t/ofXE9MdQn7i+9/50xf/3QbLTzekDdKldcgpumbP/cXln1mqqhfZC/1nh4yespYrqmZe1ps8qHP36uLEuzcvtJNWBCAxqUuzHhsYfaRvZLhQvNFYWPKbqQoiJAEIslQxCgeL/blV+ysgQJ37/3fmyntz16KES8BExiAJIOjF8suDT40Uenb5Oa26+6Jvv3rzlwyWJWQCxL5C9xeHTxRUk9YRiEbovT09PprvAYAtVwmL7NMuiLZD324seL/60Yyq90rK00PHDhb6/SD+vDl9fWFuybcZrWSBBEBO1k7v6zqQ7+6W90aAYCd7QVJWZZ4QVYNmNXABE6upTy43imphtNz9/MDJnKS7nL9x6+J4Y8IOXD8KETKPhSE73jX05OCh4XxXJOKCajJEQaJVWg/zXuPVGxcSEt1GoaiZedX0mRSJZNK3J5bv3K7Ph1HIWmsmYkoKqtGf6z7TMzqS3zsBwtv2Qq9WVJjMk8iQtD6z62h5qMcobqDwdvImJAat6DRbBgsUuo3QqwMsKdqMWysqOQSscTeIOBP3LI4jQ1ZOKkcran4v7BekArT9F4S1s1DZEjYCYs2AOwGfl+qz7tL48qSpGE0vdgPXDt0EEkDMtgBCATo3LDRNpstKXtW75VxZUWvcFSClQb0d+peW70w6S2Uj/8zQsdOV/QsofygiafryJ/M3ICtAQ8gqf9FSjBdHHsor5t7WWDCWSAqlFWfZerSNsJM3IaVrUi1Fb6QUSWrdIfuXvthgDjL3C+/zkBM5lBVTVoXG1t6faeeQz7/y3Q18oE3RDr2zLVkp7R+JhOqeW/ddAEw3B2hlHSl1Bhky0yIYWDJljQgKqvnVQ6cX/QnuJA7nacghQFSD5rLfnPcaM+7y23NXT/Qe+9X0zRnvDgZBVkWUZjCQxSBKWr6g5vJqR4tZtw0CBoYGKCOwMInTfR1KWm6DNyxt7/ESQF41Xjp4btarxSSa3AfEbF1aW2TaU+XeAIsAIIea1Fc38ul7EfbkqWRb3O3gCmmRWMa0w8o8QCREgSgQAVir9QhASIlq5PLlI+WTv5X7ii6rAGSp2v5i39eOfqliWrqiZDtCUvZYvDica9avLU2+evmnC/Vbgdv0k/YqMEj9MIlJ2W4Ce6SsM5ClaCVu5BWdMRaL5LOlyev1+XRfh11HXjVGCpWvHnjEkBWBRO1NkDBjdrBV3nmXVCEAQMwiWQ+fNR/Zrw/hbldut+vRdoHgptWCg1l6E1vbREHLo8mcQEAGIq/KlUNPfuvAi0PKgAX59HxLtU527//aoSfKRk5VlGwlUPvaBE0eNLnrRF7K+ayqEydgkGhGaOYizdjT/BQBmEx5emikRzMZYJBEacVgJOK9uC8ClLTcSKFysjJ8unvkTNfIIaurWzMkIkorGNpbQiKl+UTKKt1AYUquwspdej/rVWBn1OrdaJfWw052KEvr8Vr1DERtNw+yArqVOZEdD0RgKEpJtQ6Uu3+ne3Cf1W2tYsMYspJmPd5/pJb4/3Dzo2qzGYuEEbQLfdpW8i5kv2BWrijtO20ZRban6yQAcoqSV9W8qiIjFOCCU4NaDaoGKNJuvz4rVcDDVuUPTzzf5AECeqH37s0Lt6OgKSI78BqhHyYc02WbqQylTLpqFXPq86Ujj5mnJWlHuxPdg9XJiSe+/51td5g0WenO5w91DfIwIKIa9xqh34yC1luHMkPW0qsEBDrTepT8l0dPPb3vWJeev78cQWZSt154YeBUyKN3pz+vB64TetByrO6r5Mz0uUCM1LioyF82oE9T25vi7B1i1UhUI1YMFrsB0I/wrQPQ3Q+929sLfuOsKkPWpVtdupX+6IbecL5ic88T8a8WJt6ZuVIN7Ubop7kLBMqrRl63DpWHfnP/I8NWuSiZGmy5KmFjtJMT51/57jYFiABUJj/Zc/REYV8UxwA0aS//fPJyUwTNyG+Evh16It1hkgAABZNAMwtS+UtDZ14YPVzJFdZzOVUmV/TC1/d/4Wz3gb+/9eGCW6+Fns09P+aZfqPMUBKmYQUKFgsDpX3XeqyTvYAMOlrOsRPkFT3ZdzJfm3UDz0/8Ju9xOCSKWF070OE+KVvdDC6nmTnNHAQIk2jY6j7bu/+/Xf3F1cZsKGKLqTlZK+jmS2NfOFoeLGu5jvY22TrarAQAYLgbL5NKSNiBV+eeGwXzXv1vb56f8+qqKidEAY9NVEulYmD2HRo8/c1ceUjr6D2MNvfs0F8OnM+r0z+bvLjo2w0exK1tPVLRISAZ5YJuaCP1Xz945kXzqQHsl3fV3q+JGsCN+tyfXfzJ9PJkXXgHi5U/OPmlZ3qP6yx7L+ddYrE+97PaHIwWer790Fe2GtA0o2DCWfzB9fO6JI8UKwqTznYdqBiFomrenyzbLayeG7tjsyVkZcMqGxYC9LqFm/Wpgd78c11nphpLH1VvPjlwYqw4JOlWQTG6ZK1DfVpQzYJq9uVKQ1bXie59P7nz6ceLt+uBa3OPso2oSAK5rFpne0ZeGj3TpxVLWHwA0gMAZYAx3fra8In/Yi/YridzlPlKtntLL+zZ4UYolqIPW93/9uSLXMS6rFiKrjBpq7UlW8VqKd99p89UtJcOPw4AqqQcLQw9N3xKleSyZqWvaNwqFCZVjIKpaL9rPP3kwNG/v/lRlTteEsSJAKC8ZB7K7/vakbODZtnYev3httHgniAShJZmzXu2QOme5YWdi8VOSOoURdUsPpAXNK+J3TFhDwZ26De42+BeEPNLy3e69PyBfF+XkS+opqXu+bu320gVzN9e/+BIaeSThWknrvcb1m8efOxkZURmjDq2X+2rpV8e8Cvodwv/nASoDT/m6SZ5hqxtwP/uBdrmSRCVtfy3jn9RQswpSppDbW+Z9c9dLDrHP0sB+sfFE9//Tvrl/Cvf/cdtyT8F/D9Xsbv0RlB1EgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<PIL.Image.Image image mode=RGB size=192x64>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 128\n",
    "ds_train= CaptchaDataset(characters, 100 * batch_size, \n",
    "                         width, height, seq_length, txt_length)\n",
    "\n",
    "ds_val = CaptchaDataset(characters, 20 * batch_size, \n",
    "                        width, height, seq_length, txt_length)\n",
    "\n",
    "dl_train = DataLoader(ds_train, batch_size=batch_size, num_workers=0)\n",
    "dl_val = DataLoader(ds_val, batch_size=batch_size, num_workers=0)\n",
    "\n",
    "ds_test = CaptchaDataset(characters, 1, width, height, seq_length, txt_length)\n",
    "\n",
    "image, target, input_length, label_length = ds_test[0]\n",
    "print(''.join([characters[x] for x in target]), input_length, label_length)\n",
    "to_pil_image(image)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  二，定义模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-18T11:19:45.744324Z",
     "start_time": "2019-06-18T11:19:45.738366Z"
    }
   },
   "outputs": [],
   "source": [
    "class CRNN(nn.Module):\n",
    "    def __init__(self, n_classes, input_shape=(3, 64, 128)):\n",
    "        super().__init__()\n",
    "        self.input_shape = input_shape\n",
    "        channels = [32, 64, 128, 256, 256]\n",
    "        layers = [2, 2, 2, 2, 2]\n",
    "        kernels = [3, 3, 3, 3, 3]\n",
    "        pools = [2, 2, 2, 2, (2, 1)]\n",
    "        modules = OrderedDict()\n",
    "        \n",
    "        def cba(name, in_channels, out_channels, kernel_size):\n",
    "            modules[f'conv{name}'] = nn.Conv2d(in_channels, out_channels, kernel_size,\n",
    "                                               padding=(1, 1) if kernel_size == 3 else 0)\n",
    "            modules[f'bn{name}'] = nn.BatchNorm2d(out_channels)\n",
    "            modules[f'relu{name}'] = nn.ReLU(inplace=True)\n",
    "        \n",
    "        last_channel = 3\n",
    "        for block, (n_channel, n_layer, n_kernel, k_pool) in enumerate(zip(channels, layers, kernels, pools)):\n",
    "            for layer in range(1, n_layer + 1):\n",
    "                cba(f'{block+1}{layer}', last_channel, n_channel, n_kernel)\n",
    "                last_channel = n_channel\n",
    "            modules[f'pool{block + 1}'] = nn.MaxPool2d(k_pool)\n",
    "        modules[f'dropout'] = nn.Dropout(0.25, inplace=True)\n",
    "        \n",
    "        self.cnn = nn.Sequential(modules)\n",
    "        self.lstm = nn.LSTM(input_size=self.infer_features(), hidden_size=128, num_layers=2, bidirectional=True)\n",
    "        self.fc = nn.Linear(in_features=256, out_features=n_classes)\n",
    "    \n",
    "    def infer_features(self):\n",
    "        x = torch.zeros((1,)+self.input_shape)\n",
    "        x = self.cnn(x)\n",
    "        x = x.reshape(x.shape[0], -1, x.shape[-1])\n",
    "        return x.shape[1]\n",
    "    \n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.cnn(x)\n",
    "        x = x.reshape(x.shape[0], -1, x.shape[-1])\n",
    "        x = x.permute(2, 0, 1)\n",
    "        x, _ = self.lstm(x)\n",
    "        x = self.fc(x)\n",
    "        return x\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([12, 32, 37])\n"
     ]
    }
   ],
   "source": [
    "net = CRNN(n_classes, input_shape=(3, height, width))\n",
    "inputs = torch.zeros((32, 3, height, width))\n",
    "outputs = net(inputs)\n",
    "print(outputs.shape) # LSTM默认输出的形状是 Length在前\n",
    "#net.cuda();\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三， 训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-18T11:19:48.052899Z",
     "start_time": "2019-06-18T11:19:48.041088Z"
    }
   },
   "outputs": [],
   "source": [
    "# 解码函数和计算准确率函数\n",
    "\n",
    "def decode_target(sequence):\n",
    "    return ''.join([characters[x] for x in sequence]).replace(' ', '')\n",
    "\n",
    "def decode(sequence):\n",
    "    a = ''.join([characters[x] for x in sequence])\n",
    "    s = ''.join([x for j, x in enumerate(a[:-1]) if x != characters[0] and x != a[j+1]])\n",
    "    if len(s) == 0:\n",
    "        return ''\n",
    "    if a[-1] != characters[0] and s[-1] != a[-1]:\n",
    "        s += a[-1]\n",
    "    return s\n",
    "\n",
    "\n",
    "def eval_acc(targets, preds):\n",
    "    preds_argmax = preds.detach().permute(1, 0, 2).argmax(dim=-1)\n",
    "    targets = targets.cpu().numpy()\n",
    "    preds_argmax = preds_argmax.cpu().numpy()\n",
    "    a = np.array([decode_target(gt) == decode(pred) for gt,\n",
    "                  pred in zip(targets, preds_argmax)])\n",
    "    return a.mean()\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn.functional as F \n",
    "from torchkeras import KerasModel\n",
    "\n",
    "#我们覆盖KerasModel的StepRunner以实现自定义训练逻辑。\n",
    "#注意这里把acc指标的结果写在了step_loss中以便和loss一样在Epoch上求平均，这是一个非常灵活而且有用的写法。\n",
    "\n",
    "class StepRunner:\n",
    "    def __init__(self, net, loss_fn, accelerator, stage = \"train\", metrics_dict = None, \n",
    "                 optimizer = None, lr_scheduler = None\n",
    "                 ):\n",
    "        self.net,self.loss_fn,self.metrics_dict,self.stage = net,loss_fn,metrics_dict,stage\n",
    "        self.optimizer,self.lr_scheduler = optimizer,lr_scheduler\n",
    "        self.accelerator = accelerator\n",
    "        if self.stage=='train':\n",
    "            self.net.train() \n",
    "        else:\n",
    "            self.net.eval()\n",
    "    \n",
    "    def __call__(self, batch):\n",
    "        \n",
    "        images, targets, input_lengths, target_lengths = batch\n",
    "        \n",
    "        #loss\n",
    "        preds = self.net(images)\n",
    "        preds_log_softmax = F.log_softmax(preds, dim=-1)\n",
    "        loss = F.ctc_loss(preds_log_softmax, targets, input_lengths, target_lengths)\n",
    "        acc = eval_acc(targets,preds)\n",
    "            \n",
    "\n",
    "        #backward()\n",
    "        if self.optimizer is not None and self.stage==\"train\":\n",
    "            self.accelerator.backward(loss)\n",
    "            self.optimizer.step()\n",
    "            if self.lr_scheduler is not None:\n",
    "                self.lr_scheduler.step()\n",
    "            self.optimizer.zero_grad()\n",
    "            \n",
    "            \n",
    "        all_loss = self.accelerator.gather(loss).sum()\n",
    "        \n",
    "        #losses\n",
    "        step_losses = {self.stage+\"_loss\":\n",
    "                       all_loss.item(),\n",
    "                       self.stage+'_acc':acc}\n",
    "        \n",
    "        #metrics\n",
    "        step_metrics = {}\n",
    "        if self.stage==\"train\":\n",
    "            if self.optimizer is not None:\n",
    "                step_metrics['lr'] = self.optimizer.state_dict()['param_groups'][0]['lr']\n",
    "            else:\n",
    "                step_metrics['lr'] = 0.0\n",
    "        return step_losses,step_metrics\n",
    "    \n",
    "    \n",
    "KerasModel.StepRunner = StepRunner \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-18T12:18:50.675432Z",
     "start_time": "2019-06-18T11:19:48.053976Z"
    }
   },
   "outputs": [],
   "source": [
    "model = KerasModel(net,\n",
    "                   loss_fn=None,\n",
    "                   optimizer = torch.optim.AdamW(net.parameters(),lr = 2e-6)\n",
    "                   )\n",
    "\n",
    "#model.load_ckpt('ctc_crnn.pt')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-18T12:18:50.675432Z",
     "start_time": "2019-06-18T11:19:48.053976Z"
    },
    "code_folding": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m<<<<<< 🐌 cpu is used >>>>>>\u001b[0m\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAGJCAYAAABy9cILAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABJr0lEQVR4nO3dd1gUV9sG8HtZYClSBKkKiIIdNIoilhDFN8QWewsqlsT4ib03rIlYohEr+qYYe9dEjRrsGtEgirFiL1EBGyDS2fP9wcvGDb257OT+XddeumfOzD5nC3szM2eQCSEEiIiIiLScjqYLICIiIioNDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNaRVZs2aBZlMhhcvXmi6lPfmwYMHkMlkWLdunaZLIQ1buHAhatWqBaVSqelS3pv39Zl/+fIljI2N8euvv5bp41DZYqghKoR58+Zh7969mi7jXy81NRWTJk2Cvb09DA0N4enpidDQ0EKv/+TJE/Ts2RPm5uYwNTVFp06dcO/evVz7fv/996hduzYMDAzg6uqK5cuXv7dt5iYhIQELFizApEmToKNT8I/ulJQUpKenF3r7UhUVFYUxY8agWbNmMDAwgEwmw4MHD3L0s7S0xOeff47AwMD3XySVGoYaokJgqCkfBgwYgCVLlsDPzw/BwcGQy+Vo164dzpw5U+C6iYmJaNWqFU6ePImpU6di9uzZuHTpEry9vfHy5Uu1vmvWrMHnn3+OunXrYvny5fDy8sLIkSOxYMGCMt9mXn744QdkZGSgT58+efYJDw9Hv379YGNjA0NDQygUCjg4OGDkyJG4c+dOoR5HasLCwrBs2TK8efMGtWvXzrfv0KFDcfHiRRw7duw9VUelThBpkZkzZwoA4vnz5+/1cY2NjYW/v/97fcxs9+/fFwDEjz/+qJHHLy/Onz8vAIhFixap2pKTk0X16tWFl5dXgesvWLBAABB//PGHqu3GjRtCLpeLKVOmqNqSkpKEpaWlaN++vdr6fn5+wtjYWLx69apMt5kXd3d30bdv31yXpaeni4CAACGTyUTLli3FN998I/bt2yd27dol5s2bJxo0aCAMDAzEihUrCnyc8qakn/mXL1+KhIQEIYQQixYtEgDE/fv38+xfr1490a9fv2I9FmkeQw1plewfcDdu3BA9evQQJiYmwsLCQowcOVIkJyfn6L9hwwbRsGFDYWBgICpWrCh69eolHj16pNbn1q1bomvXrsLGxkYoFApRuXJl0atXLxEXFyeEEAJAjlteASc6OlrI5XIxa9asHMtu3rwpAIjly5cLIbJ+2I4bN07Uq1dPGBsbCxMTE/HJJ5+IyMhItfWKE2pSU1NFYGCgaNiwoTA1NRVGRkaiRYsW4tixYzn6ZmZmiqVLl4p69eoJhUIhKlWqJHx9fUV4eLhavw0bNojGjRsLQ0NDYW5uLlq2bCkOHz5c6JpKasKECUIul4v4+Hi19nnz5gkAOV7Xf2rcuLFo3LhxjvaPP/5YVK9eXXX/wIEDAoA4cOCAWr+zZ88KAGLDhg1lus3c3Lt3TwAQ69aty3V53759RcWKFcXBgwfz3Ma6deuEgYGBWL16dY5lf/31lxg4cKCwtrYW+vr6ok6dOuL7779X63P8+HEBQGzdulVMmTJF2NjYCCMjI9GxY8dcn/vt27erPnuWlpbCz89P/PXXXzn6ZX+WK1WqJAwMDESNGjXE1KlTVcuzP/O3b98W/v7+wszMTJiamooBAwaIt2/f5jne3BQm1IwZM0aYm5sLpVJZpG1T+cDDT6SVevbsiZSUFAQFBaFdu3ZYtmwZhgwZotbn66+/Rv/+/eHq6oolS5Zg9OjROHr0KD788EPExcUBANLS0uDr64tz585hxIgRWLlyJYYMGYJ79+6p+mzYsAEKhQItW7bEhg0bsGHDBnz55Ze51mVjYwNvb29s3749x7Jt27ZBLpejR48eAIB79+5h79696NChA5YsWYIJEybgypUr8Pb2xtOnT0v0/CQkJOC7777DRx99hAULFmDWrFl4/vw5fH19ERkZqdZ38ODBGD16NBwcHLBgwQJMnjwZBgYGOHfunKrP7Nmz0a9fP+jp6WHOnDmYPXs2HBwcCtxNn5qaihcvXhTqVpBLly6hRo0aMDU1VWtv0qQJAOQY17uUSiX+/PNPeHh45FjWpEkT3L17F2/evFE9DoAcfRs1agQdHR3V8rLYZl7Onj0LAGjYsGGOZRs2bMCePXtw+vRpfPLJJwAAIQQSExNVfV68eIF+/fph586dGDt2LB4+fKhaFhMTg6ZNm+LIkSMYPnw4goOD4eLigsGDB2Pp0qU5Hu/rr7/GgQMHMGnSJIwcORKhoaFo06YNkpOTVX3WrVuHnj17Qi6XIygoCF988QV2796NFi1aqD5XAPDnn3/C09MTx44dwxdffIHg4GB07twZ+/bty/G4PXv2xJs3bxAUFISePXti3bp1mD17dr7PW3E0atQIcXFxuHbtWqlvm94DTacqoqLI/q3t008/VWsfNmyYACAuX74shBDiwYMHQi6Xi6+//lqt35UrV4Surq6q/dKlSwKA2LFjR76PW5TDT2vWrBEAxJUrV9Ta69SpI1q3bq26n5KSIjIzM9X63L9/XygUCjFnzhy1NhRxT01GRoZITU1Va3v9+rWwsbERgwYNUrUdO3ZMABAjR47MsY3s31Rv374tdHR0RJcuXXLUW9Bvsz/++GOue7pyuxWkbt26as9ftmvXrgkAIiQkJM91nz9/LgCoPa/ZVq5cKQCImzdvCiGECAgIEHK5PNftWFlZid69e5fZNvMyffp0AUC8efNGrV2pVApnZ2exdOlSVdvPP/8s7O3tBQDh6OgoDh8+rLZ3okuXLmp7QgYPHizs7OzEixcv1Lbdu3dvYWZmJpKSkoQQf++pqVy5supwjhBZe2QAiODgYCGEEGlpacLa2lrUq1dPbe/p/v37BQAxY8YMVduHH34oTExMxMOHD3OMK1v2Z/7d9232OCwtLfN93v6pMHtqsveebdu2rUjbpvKBe2pIKwUEBKjdHzFiBACopmPu3r0bSqUSPXv2VNsbYGtrC1dXVxw/fhwAYGZmBgA4fPgwkpKSSqW2rl27QldXF9u2bVO1Xb16FdevX0evXr1UbQqFQjWLJTMzEy9fvkSFChVQs2ZNXLx4sUQ1yOVy6OvrA8jao/Dq1StkZGTAw8NDbdu7du2CTCbDzJkzc2xDJpMBAPbu3QulUokZM2bkmHWT3Scvvr6+CA0NLdStIMnJyVAoFDnaDQwMVMvzWxdAodZPTk5WPXe59X23X2lvMy8vX76Erq4uKlSooNYeERGB2NhYDB48GEDWTKw+ffqgSZMm2LVrF8aMGYNBgwaprdO5c2ecOHECQNYenV27dqFjx44QQqh9Vnx9fREfH5/jvdi/f3+YmJio7nfv3h12dnaqz96FCxcQGxuLYcOGqZ4HAGjfvj1q1aqFAwcOAACeP3+OU6dOYdCgQXB0dFR7jNzeV0OHDlW737JlS7x8+RIJCQn5PndFVbFiRQD4V102Qkp0NV0AUXG4urqq3a9evTp0dHRUUzVv374NIUSOftn09PQAAM7Ozhg7diyWLFmCTZs2oWXLlvj000/Rt29fVeApqkqVKsHHxwfbt2/H3LlzAWQdetLV1UXXrl1V/ZRKJYKDg7Fq1Srcv38fmZmZqmWWlpbFeux3/fTTT1i8eDFu3rypNrXX2dlZ9f+7d+/C3t4eFhYWeW7n7t270NHRQZ06dYpcg52dHezs7Iq8Xm4MDQ2Rmpqaoz0lJUW1PL91ARRqfUNDQ6SlpeW6nZSUFLV+pb3NooqIiICHh4cq7GzatAmVK1fGzp07IZfLAQDm5uYYOHCgah0bGxs8f/4cQFawiIuLw9q1a7F27dpcHyM2Nlbt/j8/UzKZDC4uLqrPXvahrZo1a+bYVq1atVQz1bKnvderV69QY/1n8MkOH69fv85xSLIkhBAACg7sVD4x1JAk/PMHkFKphEwmw8GDB1U/3N/17m+8ixcvxoABA/Dzzz/jt99+w8iRIxEUFIRz586hSpUqxaqnd+/eGDhwICIjI9GgQQNs374dPj4+qFSpkqrPvHnzEBgYiEGDBmHu3LmwsLCAjo4ORo8eXeKLq23cuBEDBgxA586dMWHCBFhbW6vOb7h7926Jtl0UycnJiI+PL1RfW1vbfJfb2dnhyZMnOdqfPXsGALC3t89zXQsLCygUClXf/Na3s7NDZmYmYmNjYW1treqXlpaGly9fqvqVxTbzYmlpiYyMDLx580ZtL8k/133w4AE++OADtfd89jlH2R4/fqwKzdnvs759+8Lf3z/Xx3Z3d8+3tvclt88x8HcIKS2vX78GALXPKmkPhhrSSrdv31bb43Dnzh0olUpUrVoVQNaeGyEEnJ2dUaNGjQK35+bmBjc3N0yfPh1nz55F8+bNERISgq+++gpA0X9r69y5M7788kvVIahbt25hypQpan127tyJVq1a4fvvv1drj4uLK/EP1J07d6JatWrYvXu3Wu3/PMxUvXp1HD58GK9evcpzb0316tWhVCpx/fp1NGjQoEh1bNu2TW0vQX4K+nJq0KABjh8/joSEBLXfzM+fP69anhcdHR24ubnhwoULOZadP38e1apVU4WF7O1cuHAB7dq1U/W7cOEClEqlanlZbDMvtWrVAgDcv39fLWSYmpqqhUZbW1v88ccfauu+eyFAIQS+//57tGnTBgBgZWUFExMTZGZmqtoKcvv2bbX7QgjcuXNHVZeTkxOArIvetW7dWq1vVFSUanm1atUAZB2aLU/u378PAAVe04bKJ55TQ1pp5cqVavezr8zatm1bAFnntcjlcsyePTvHl6UQQnVhtISEBGRkZKgtd3Nzg46OjtphBWNjY7VZGwUxNzeHr68vtm/fjq1bt0JfXx+dO3dW6yOXy3PUtmPHjlz3RhRV9m+1727//PnzCAsLU+vXrVs3CCFynUWSvW7nzp2ho6ODOXPm5NiDVFAQKc1zarp3747MzEy1wySpqan48ccf4enpCQcHB1X7o0ePcPPmzRzrh4eHq4WQqKgoHDt2TDUjDQBat24NCwsLrF69Wm391atXw8jICO3bty/TbebGy8sLAHIEqNq1ayM8PFz1unTq1AmXLl3CjBkzcO/ePZw+fRoTJkwAkDUDq1u3bvjrr78watQoAFnvk27dumHXrl25hovsw1TvWr9+vWpWF5AVoJ89e6b67Hl4eMDa2hohISFqn6GDBw/ixo0bqrFaWVnhww8/xA8//IBHjx6pPUZp730pioiICJiZmaFu3boaq4FK4L2fmkxUAtkzIdzc3ETHjh3FypUrRd++fQUA8dlnn6n1DQoKEgBEs2bNxMKFC8Xq1avFxIkThaurq+oCbnv27BGVK1cWo0ePFqtWrRLLli0TjRs3Fnp6eiIsLEy1rXbt2gljY2OxePFisWXLFnHu3LkCa924caMAIExMTETHjh1zLJ8xY4YAIAYMGCDWrl0rRowYISwsLES1atWEt7e3ql9xZj/98MMPqllia9asEZMnTxbm5uaibt26wsnJSa1vv379BADRtm1bERwcLL799lvRtWtX1fV0hBAiMDBQ9Vx+8803Yvny5aJ///5i8uTJha6pNPTo0UPo6uqKCRMmiDVr1ohmzZoJXV1dcfLkSbV+3t7eOWZUJSQkiOrVqwtra2uxcOFC8e233woHBwdhb28vYmNj1fpmz17q3r27+O9//yv69+8vAOSYTVcW28xLvXr1RJ8+fdTaUlJShJmZmdizZ4+qbd68eUJHR0cAELq6uiI4OFg1w+zjjz8W9+7dU9tGdHS0cHJyEkZGRmLUqFFizZo1IigoSPTo0UNUrFhR1S979pObm5twd3cX3377rZg8ebIwMDAQLi4uateMyZ715unpKZYuXSqmTJkijIyMRNWqVcXr169V/SIjI0WFChWEpaWlmDJlili7dq2YOnWqqF+/vqpPXhffy36M/GYyCSFEXFycmDt3rpg7d6745JNPBAAxbtw4MXfuXLX3+LvPc14XOaTyj6GGtEr2D7jr16+L7t27CxMTE1GxYkUxfPjwXC++t2vXLtGiRQthbGwsjI2NRa1atURAQICIiooSQmRd1GzQoEGievXqwsDAQFhYWIhWrVqJI0eOqG3n5s2b4sMPPxSGhob5XnzvXQkJCar+GzduzLE8JSVFjBs3TtjZ2QlDQ0PRvHlzERYWJry9vUscapRKpZg3b55wcnISCoVCfPDBB2L//v3C398/R6jJyMgQixYtErVq1RL6+vrCyspKtG3bVkRERKj1++GHH8QHH3wgFAqFqFixovD29hahoaGFrqk0JCcni/HjxwtbW1uhUChE48aNxaFDh3L0yy3UCCHE48ePRffu3YWpqamoUKGC6NChg7h9+3auj7V27VpRs2ZNoa+vL6pXry6+/fbbXKewl8U2c7NkyRJRoUIF1RTrbDNnzhTVqlVTuyrxkydPxKlTp0R0dLQQQogzZ87kCFnviomJEQEBAcLBwUHo6ekJW1tb4ePjI9auXavqkx1qtmzZIqZMmSKsra2FoaGhaN++fY4p2UIIsW3bNtX7xcLCIs+L7129elV06dJFmJubCwMDA1GzZk0RGBioNr6ShJrsz09ut39+Fm7cuCEA5Pj8k/aQCaHB/XxERFQo8fHxqFatGhYuXKiawg1kzZ5q3rw55HI5fv755zxnm+3cuRNdunTJ84Tbgpw4cQKtWrXCjh070L1792Jto7wbPXo0Tp06hYiICM5+0lI8p4aISAuYmZlh4sSJWLRokdq5TQYGBvj1118hk8lQs2ZNTJo0CadOncLDhw9x8+ZNrF+/Hl5eXvD39y/x9Y+k7OXLl/juu+/w1VdfMdBoMe6pIdIiaWlpePXqVb59zMzMin3dE9JeaWlpWLFiBVasWKGawQNkhZ4uXbpg9uzZeV63qTD+DXtqSPtxSjeRFjl79ixatWqVb58ff/wRAwYMeD8FUbmhr6+PsWPHYuzYsXjw4AGePHkCAwMD1K5dG0ZGRpouj+i94J4aIi3y+vVrRERE5Nunbt26pXYVXyIibcJQQ0RERJLAE4WJiIhIEnhOTSlQKpV4+vQpTExMeNY8ERFREQgh8ObNG9jb20NHp2T7WhhqSsHTp0/VLtFORERERfP48eNi/xHhbAw1pSD7j9Y9fvxY7Q/tERERUf4SEhLg4OCg9hfoi4uhphRkH3IyNTVlqCEiIiqG0jh9gycKExERkSQw1BAREZEkMNQQERGRJPCcGiIiKveEEMjIyEBmZqamS6Eiksvl0NXVfS+XPGGoISKici0tLQ3Pnj1DUlKSpkuhYjIyMoKdnR309fXL9HEYaoiIqNxSKpW4f/8+5HI57O3toa+vz4ucahEhBNLS0vD8+XPcv38frq6uJb7AXn4YaoiIqNxKS0uDUqmEg4MD/9q4ljI0NISenh4ePnyItLQ0GBgYlNlj8URhIiIq98ryt3sqe+/r9eO7hIiIiCSBoYaIiIgkgaGGiIj+FTIzgRMngC1bsv7VptnhVatWxdKlSzVdRrnHE4WJiEjydu8GRo0C/vrr77YqVYDgYKBr17J5zI8++ggNGjQolTASHh4OY2PjkhclcdxTQ0REkrZ7N9C9u3qgAYAnT7Lad+/WTF3ZFxQsDCsrK87+KgSGGiIi0kpv3+Z9S0nJ6pOZmbWHRoic62e3jRqlfigqr20WxYABA3Dy5EkEBwdDJpNBJpNh3bp1kMlkOHjwIBo1agSFQoEzZ87g7t276NSpE2xsbFChQgU0btwYR44cUdvePw8/yWQyfPfdd+jSpQuMjIzg6uqKX375pVC1ZWZmYvDgwXB2doahoSFq1qyJ4ODgHP1++OEH1K1bFwqFAnZ2dhg+fLhqWVxcHL788kvY2NjAwMAA9erVw/79+4v2JJUBhhoiItJKFSrkfevWLavP6dM599C8S4is5adP/91WtWru2yyK4OBgeHl54YsvvsCzZ8/w7NkzODg4AAAmT56M+fPn48aNG3B3d0diYiLatWuHo0eP4tKlS/jkk0/QsWNHPHr0KN/HmD17Nnr27Ik///wT7dq1g5+fH169elVgbUqlElWqVMGOHTtw/fp1zJgxA1OnTsX27dtVfVavXo2AgAAMGTIEV65cwS+//AIXFxfV+m3btsXvv/+OjRs34vr165g/fz7kcnnRnqSyIKjE4uPjBQARHx+v6VKIiCQlOTlZXL9+XSQnJ+dYlhVJcr+1a5fVZ/Pm/Ptl3zZv/nu7lSrl3qeovL29xahRo1T3jx8/LgCIvXv3Frhu3bp1xfLly1X3nZycxLfffvvO2CGmT5+uup+YmCgAiIMHDxa9UCFEQECA6Natm+q+vb29mDZtWq59Dx8+LHR0dERUVFSht5/f61ia36E8UZiIiLRSYmLey7J3GtjZFW5b7/Z78KDYJRWKh4eH2v3ExETMmjULBw4cwLNnz5CRkYHk5OQC99S4u7ur/m9sbAxTU1PExsYWqoaVK1fihx9+wKNHj5CcnIy0tDQ0aNAAABAbG4unT5/Cx8cn13UjIyNRpUoV1KhRo1CP9T4x1BARkVYqzGSgli2zZjk9eZL7eTUyWdbyli2Ltt2S+OcspvHjxyM0NBTffPMNXFxcYGhoiO7duyMtLS3f7ejp6andl8lkUCqVBT7+1q1bMX78eCxevBheXl4wMTHBokWLcP78eQBZf9YgPwUt1ySeU0NERJIll2dN2wayAsy7su8vXfr3np3SpK+vj8xCXAzn999/x4ABA9ClSxe4ubnB1tYWD8pwd9Hvv/+OZs2aYdiwYfjggw/g4uKCu3fvqpabmJigatWqOHr0aK7ru7u746+//sKtW7fKrMbiYqghIiJJ69oV2LkTqFxZvb1Klaz2srpOTdWqVXH+/Hk8ePAAL168yHMviqurK3bv3o3IyEhcvnwZn332WaH2uBSXq6srLly4gMOHD+PWrVsIDAxEeHi4Wp9Zs2Zh8eLFWLZsGW7fvo2LFy9i+fLlAABvb298+OGH6NatG0JDQ3H//n0cPHgQhw4dKrOaC4uhhoiIJK9r16xzZY4fBzZvzvr3/v2yCzRA1mEluVyOOnXqwMrKKs9zZJYsWYKKFSuiWbNm6NixI3x9fdGwYcMyq+vLL79E165d0atXL3h6euLly5cYNmyYWh9/f38sXboUq1atQt26ddGhQwfcvn1btXzXrl1o3Lgx+vTpgzp16mDixImF2itV1mRC5HaUkYoiISEBZmZmiI+Ph6mpqabLISKSjJSUFNy/fx/Ozs4wMDDQdDlUTPm9jqX5Hco9NURERCQJDDVEREQSMnToUFSoUCHX29ChQzVdXpnilG4iIiIJmTNnDsaPH5/rMqmfIsFQQ0REJCHW1tawtrbWdBkawcNPREREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVERPSvkCkETrx+jS0xMTjx+jUyy/kF9atWrYqlS5dqugytwindREQkebufP8eoO3fwV2qqqq2KQoFgFxd0tbLSYGVUmrinhoiIJG338+fofu2aWqABgCepqeh+7Rp2P3+uocqotDHUEBGRVhFC4G1mZqFuCRkZGHn7NnI70JTdNurOHSRkZBRqe4X9G9Br166Fvb09lEqlWnunTp0waNAg3L17F506dYKNjQ0qVKiAxo0b48iRI8V+TpYsWQI3NzcYGxvDwcEBw4YNQ2Jiolqf33//HR999BGMjIxQsWJF+Pr64vXr1wAApVKJhQsXwsXFBQqFAo6Ojvj666+LXY+maF2oWblyJapWrQoDAwN4enrijz/+yLf/jh07UKtWLRgYGMDNzQ2//vprnn2HDh0KmUzGY5hEROVYklKJCqdPF+pmduYMnqSl5bktAeCv1FSYnTlTqO0l/SOk5KVHjx54+fIljh8/rmp79eoVDh06BD8/PyQmJqJdu3Y4evQoLl26hE8++QQdO3bEo0ePivWc6OjoYNmyZbh27Rp++uknHDt2DBMnTlQtj4yMhI+PD+rUqYOwsDCcOXMGHTt2RGZmJgBgypQpmD9/PgIDA3H9+nVs3rwZNjY2xapFk7TqnJpt27Zh7NixCAkJgaenJ5YuXQpfX19ERUXleknos2fPok+fPggKCkKHDh2wefNmdO7cGRcvXkS9evXU+u7Zswfnzp2Dvb39+xoOERFJVMWKFdG2bVts3rwZPj4+AICdO3eiUqVKaNWqFXR0dFC/fn1V/7lz52LPnj345ZdfMHz48CI/3ujRo1X/r1q1Kr766isMHToUq1atAgAsXLgQHh4eqvsAULduXQDAmzdvEBwcjBUrVsDf3x8AUL16dbRo0aLIdWiaVoWaJUuW4IsvvsDAgQMBACEhIThw4AB++OEHTJ48OUf/4OBgfPLJJ5gwYQKArDdNaGgoVqxYgZCQEFW/J0+eYMSIETh8+DDat2//fgZDRETFYqSjg8SWLQvV91RcHNpduVJgv1/d3PChuXmhHruw/Pz88MUXX2DVqlVQKBTYtGkTevfuDR0dHSQmJmLWrFk4cOAAnj17hoyMDCQnJxd7T82RI0cQFBSEmzdvIiEhARkZGUhJSUFSUhKMjIwQGRmJHj165LrujRs3kJqaqgpf2kxrDj+lpaUhIiICbdq0UbXp6OigTZs2CAsLy3WdsLAwtf4A4Ovrq9ZfqVSiX79+mDBhgiq1FiQ1NRUJCQlqNyIiej9kMhmM5fJC3T62sEAVhQKyvLYFwEGhwMcWFoXankyW15Zy6tixI4QQOHDgAB4/fozTp0/Dz88PADB+/Hjs2bMH8+bNw+nTpxEZGQk3Nzek5XOoLC8PHjxAhw4d4O7ujl27diEiIgIrV64EANX2DA0N81w/v2XaRmtCzYsXL5CZmZnjGJ+NjQ2io6NzXSc6OrrA/gsWLICuri5GjhxZ6FqCgoJgZmamujk4OBRhJERE9L7IZTIEu7gAQI5gk31/qYsL5EUIK4VlYGCArl27YtOmTdiyZQtq1qyJhg0bAsg6aXfAgAHo0qUL3NzcYGtriwcPHhTrcSIiIqBUKrF48WI0bdoUNWrUwNOnT9X6uLu74+jRo7mu7+rqCkNDwzyXaxOtCTVlISIiAsHBwVi3bl2R0veUKVMQHx+vuj1+/LgMqyQiopLoamWFnXXrorJCodZeRaHAzrp1y/Q6NX5+fqrTJLL30gBZQWL37t2IjIzE5cuX8dlnn+WYKVVYLi4uSE9Px/Lly3Hv3j1s2LBB7RQLIOt7Kzw8HMOGDcOff/6JmzdvYvXq1Xjx4gUMDAwwadIkTJw4EevXr8fdu3dx7tw5fP/99yUauyZoTaipVKkS5HI5YmJi1NpjYmJga2ub6zq2trb59j99+jRiY2Ph6OgIXV1d6Orq4uHDhxg3bhyqVq2aZy0KhQKmpqZqNyIiKr+6WlnhQdOmOF6/PjbXro3j9evjftOmZX7hvdatW8PCwgJRUVH47LPPVO1LlixBxYoV0axZM3Ts2BG+vr6qvThFVb9+fSxZsgQLFixAvXr1sGnTJgQFBan1qVGjBn777TdcvnwZTZo0gZeXF37++Wfo6madWhsYGIhx48ZhxowZqF27Nnr16oXY2NjiD1xDZKKwk+7LAU9PTzRp0gTLly8HkHU+jKOjI4YPH57ricK9evVCUlIS9u3bp2pr1qwZ3N3dERISgpcvX+LZs2dq6/j6+qJfv34YOHAgatasWai6EhISYGZmhvj4eAYcIqJSlJKSgvv378PZ2RkGBgaaLoeKKb/XsTS/Q7Vq9tPYsWPh7+8PDw8PNGnSBEuXLsXbt29Vs6H69++PypUrqxLqqFGj4O3tjcWLF6N9+/bYunUrLly4gLVr1wIALC0tYWlpqfYYenp6sLW1LXSgISIiovJBq0JNr1698Pz5c8yYMQPR0dFo0KABDh06pDoZ+NGjR9B5Z7pds2bNsHnzZkyfPh1Tp06Fq6sr9u7dm+MaNUREROXVpk2b8OWXX+a6zMnJCdeuXXvPFZVfWnX4qbzi4SciorLBw09ZF8f75/mh2fT09ODk5PSeKyo6Hn4iIiIimJiYwMTERNNlaAWtmf1ERET/XjyooN3e1+vHUENEROWWnp4eACApKUnDlVBJZL9+2a9nWeHhJyIiKrfkcjnMzc1V10wxMjIq0sVSSbOEEEhKSkJsbCzMzc0hl8vL9PEYaoiIqFzLvmCqNl4MjrKYm5vneaHc0sRQQ0RE5ZpMJoOdnR2sra2Rnp6u6XKoiPT09Mp8D002hhoiItIKcrn8vX05knbiicJEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAlaF2pWrlyJqlWrwsDAAJ6envjjjz/y7b9jxw7UqlULBgYGcHNzw6+//qpalp6ejkmTJsHNzQ3Gxsawt7dH//798fTp07IeBhEREZUyrQo127Ztw9ixYzFz5kxcvHgR9evXh6+vL2JjY3Ptf/bsWfTp0weDBw/GpUuX0LlzZ3Tu3BlXr14FACQlJeHixYsIDAzExYsXsXv3bkRFReHTTz99n8MiIiKiUiATQghNF1FYnp6eaNy4MVasWAEAUCqVcHBwwIgRIzB58uQc/Xv16oW3b99i//79qramTZuiQYMGCAkJyfUxwsPD0aRJEzx8+BCOjo6FqishIQFmZmaIj4+HqalpMUZGRET071Sa36Fas6cmLS0NERERaNOmjapNR0cHbdq0QVhYWK7rhIWFqfUHAF9f3zz7A0B8fDxkMhnMzc3z7JOamoqEhAS1GxEREWmW1oSaFy9eIDMzEzY2NmrtNjY2iI6OznWd6OjoIvVPSUnBpEmT0KdPn3zTYlBQEMzMzFQ3BweHIo6GiIiISpvWhJqylp6ejp49e0IIgdWrV+fbd8qUKYiPj1fdHj9+/J6qJCIiorzoarqAwqpUqRLkcjliYmLU2mNiYmBra5vrOra2toXqnx1oHj58iGPHjhV4TE+hUEChUBRjFERERFRWtGZPjb6+Pho1aoSjR4+q2pRKJY4ePQovL69c1/Hy8lLrDwChoaFq/bMDze3bt3HkyBFYWlqWzQCIiIioTGnNnhoAGDt2LPz9/eHh4YEmTZpg6dKlePv2LQYOHAgA6N+/PypXroygoCAAwKhRo+Dt7Y3Fixejffv22Lp1Ky5cuIC1a9cCyAo03bt3x8WLF7F//35kZmaqzrexsLCAvr6+ZgZKRERERaZVoaZXr154/vw5ZsyYgejoaDRo0ACHDh1SnQz86NEj6Oj8vfOpWbNm2Lx5M6ZPn46pU6fC1dUVe/fuRb169QAAT548wS+//AIAaNCggdpjHT9+HB999NF7GRcRERGVnFZdp6a84nVqiIiIiudfeZ0aIiIiovww1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUlIY//1T/l4iIiN67YoWabt26YcGCBTnaFy5ciB49epS4KK3zyy9Z/+7bp9k6iIiI/sWKFWpOnTqFdu3a5Whv27YtTp06VeKitM7Bg+r/EhER0XtXrFCTmJgIfX39HO16enpISEgocVFaJSYGuHo16/9XrgCxsZqth4iI6F+qWKHGzc0N27Zty9G+detW1KlTp8RFaZXDh/O/T0RERO+FbnFWCgwMRNeuXXH37l20bt0aAHD06FFs2bIFO3bsKNUCy7vM/QeglOkAQolMHR1k7j8Aeb9+mi6rWNIyBFadicPd12moXlEfw1qYQ19Xpumyio3jKb+kNBaA4ynPpDQWKphMCCGKs+KBAwcwb948REZGwtDQEO7u7pg5cya8vb1Lu0Y1K1euxKJFixAdHY369etj+fLlaNKkSZ79d+zYgcDAQDx48ACurq5YsGCB2vlAQgjMnDkT//3vfxEXF4fmzZtj9erVcHV1zerw5EnWIaZcBJ98hYHTOgHJSTADEA9AGBpj3dd7McrbIveCbGyAypWLOfqyM3HfcyxJu4NMy1RVm/ylAmP1XbCwo5UGKysejqf8ktJYAI6nPJPSWKQsISEBZmZmiI+Ph6mpaYm2VexQownbtm1D//79ERISAk9PTyxduhQ7duxAVFQUrK2tc/Q/e/YsPvzwQwQFBaFDhw7YvHkzFixYgIsXL6JevXoAgAULFiAoKAg//fQTnJ2dERgYiCtXruD69eswMDAAfHyAY8fyrEkpkyFRCFWoqSCTQSe/p9THBzhypITPROmauO85FlW4lnXn3V9glFn3JyTW1aofABxP+SWlsQAcT3kmpbFIncZDTXh4OJRKJTw9PdXaz58/D7lcDg8PjxIVlRdPT080btwYK1asAAAolUo4ODhgxIgRmDx5co7+vXr1wtu3b7F//35VW9OmTdGgQQOEhIRACAF7e3uMGzcO48ePBwDEx8fDxsYG69atQ+/evYEdO4AhQ4C4uDzrSgBUoSavl0MAgLk5UkNCkNC2e57b0tEBDA3/vv/2bZ5dc/RNSgLyejVlMsDIKGfftAwBpxN/QFkxTf2Dn00J6LxW4GGrxnnusjU2/vv/ycmAUpl3ze/2TUkBMjNLp6+RUdYY0zIEKh3Ifzzy1wo875A1ntRUICMj7+0aGmY9zwCQlgakp5dOXwMDQC4vuG9RXx+FAtD930Hl9PSsbefl3b4ZGUBqat599fUBPb2i983MzHrtijOWf9LTy9o2kPUeS07Ou4ai9NXVzXougKzPRFJS4fqmpgtY/Vr491p+n2W5POs9ka0on/vS+hmRningeLxwr49CT6b286Qon/v38TOiMO81+WsFkjo15aGockDjoaZJkyaYOHEiundX/3LevXs3FixYgPPnz5eoqNykpaXByMgIO3fuROfOnVXt/v7+iIuLw88//5xjHUdHR4wdOxajR49Wtc2cORN79+7F5cuXce/ePVSvXh2XLl1CgwYNVH28vb3RoEEDBAcHZzXExgJDhwJ79mR9c/7jKcsv1Cj/t+dmV8uW+L8xY/C8YsWSPA1ERFRKvkV9jP6IP5M1rTRDTbFmP12/fh0NGzbM0f7BBx/g+vXrJSooLy9evEBmZiZsbGzU2m1sbBAdHZ3rOtHR0fn2z/63wG1aWwO7dwPbtgFmZsiQFe5py9DRQZyxMXrOmIHuc+Yw0BARlSN3X+ezK5O0UrFmPykUCsTExKBatWpq7c+ePYOubrE2qR169gQ++giPOvSBc/ixXPdqZhMAHjRsBbOtG7Da1ho/anjXcl6Hn9aExWGG4ko+I8kyJ9UNX3qZ57qsPB1+WnE6DpNR8Hjmww3DW5qX+8NPRX19yvPhp5K+18rb4aflp+IwRVb491p5P/y09lwcAvUL9/oMbWZerg8/Ffa9Vr1izuutkXYrVgL5+OOPMWXKFPz8888wMzMDAMTFxWHq1Kn4z3/+U6oFZqtUqRLkcjli/jETKSYmBra2trmuY2trm2//7H9jYmJgZ2en1ufdw1FqrK3h9B8vZEachK4y709bpo4czr5ekFe3y7HMuAh714rU16TofSf5WGD2zwpkVkzNfb/d/449T+pkUahjz8YVilCDccF9itp3TAsLTCvEeMZ0soC+XAZjo1z65FWDIQDDAruVat8SvT5yAAa5rJMbOQBF2fQ1/d/3Rqm+1+SAiV7hayh0XwAVCvmZG9vSAtOL8l4rq899Kf2MmNjaArOK+foU6XP/Hn5GFPa9NqyTeeEfgLRCsQ4/ffPNN3j8+DGcnJzQqlUrtGrVCs7OzoiOjsbixYtLu0YAgL6+Pho1aoSjR4+q2pRKJY4ePQovL69c1/Hy8lLrDwChoaGq/s7OzrC1tVXrk5CQgPPnz+e5TQCQH9gPeT6BBgDkykzID+zPt095oK8rw1h9l6yT6f7529P/ZgmM1XfRmpPpOJ7yS0pjATie8kxKY6GiKfaU7rdv32LTpk24fPmy6jo1ffr0gZ5eEX4lKqJt27bB398fa9asQZMmTbB06VJs374dN2/ehI2NDfr374/KlSsjKCgIQNaUbm9vb8yfPx/t27fH1q1bMW/evBxTuufPn682pfvPP//8e0r3P0VHA3bqe1/iAFQE8BqAeW79/3HOTnkktes5cDzll5TGAnA85ZmUxiJlGp/9lO369et49OgR0v5x4P7TTz8tUVH5WbFiheriew0aNMCyZctUU8s/+ugjVK1aFevWrVP137FjB6ZPn666+N7ChQtzvfje2rVrERcXhxYtWmDVqlWoUaNG7gX89BMwYMDf68vleG5gDJu3CYgxNoVVylvI3j0I/NNPQP/+pfkUlBmpXXmT4ym/pDQWgOMpz6Q0FqnSeKi5d+8eunTpgitXrkAmk0EIAZns7zdJZn5ndmm7Xr2AnTuzzrYTAujSBQmLFsHMxQXxd+7AdMKEv6d+y2RAjx7A1q2arpqIiKhc0viU7lGjRsHZ2RmxsbEwMjLC1atXcfLkSXh4eODEiRMlKqhcy8gADh3KOnXfzCxrivfu3YDV/3ZjWlmpTf2GUgkcPJj/6ftERERUKoo1+yksLAzHjh1DpUqVoKOjA7lcjhYtWiAoKAgjR47EpUuXSrvO8iE5GahWDXB2BkJCsq5fk5v/Tf3G0KHAgwdZ8yhNijA1iYiIiIqsWKEmMzMTJv/7kq5UqRKePn2KmjVrwsnJCVFRUaVaYLliYgJcuPD3BUbyk33BvszMwvUnIiKiEilWqKlXrx4uX74MZ2dneHp6YuHChdDX18fatWtzXJBPcooaUBhoiIiI3otihZrp06fj7f8uYzlnzhx06NABLVu2hKWlJbZt21aqBRIREREVRommdL/r1atXqFixotosqH+L0jxzm4iI6N+kNL9DS+0PNVlYWJTWpoiIiIiKrFhTuomIiIjKG4YaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgStCTWvXr2Cn58fTE1NYW5ujsGDByMxMTHfdVJSUhAQEABLS0tUqFAB3bp1Q0xMjGr55cuX0adPHzg4OMDQ0BC1a9dGcHBwWQ+FiIiIyoDWhBo/Pz9cu3YNoaGh2L9/P06dOoUhQ4bku86YMWOwb98+7NixAydPnsTTp0/RtWtX1fKIiAhYW1tj48aNuHbtGqZNm4YpU6ZgxYoVZT0cIiIiKmUyIYTQdBEFuXHjBurUqYPw8HB4eHgAAA4dOoR27drhr7/+gr29fY514uPjYWVlhc2bN6N79+4AgJs3b6J27doICwtD06ZNc32sgIAA3LhxA8eOHSt0fQkJCTAzM0N8fDxMTU2LMUIiIqJ/p9L8DtWKPTVhYWEwNzdXBRoAaNOmDXR0dHD+/Plc14mIiEB6ejratGmjaqtVqxYcHR0RFhaW52PFx8fDwsIi33pSU1ORkJCgdiMiIiLN0opQEx0dDWtra7U2XV1dWFhYIDo6Os919PX1YW5urtZuY2OT5zpnz57Ftm3bCjysFRQUBDMzM9XNwcGh8IMhIiKiMqHRUDN58mTIZLJ8bzdv3nwvtVy9ehWdOnXCzJkz8fHHH+fbd8qUKYiPj1fdHj9+/F5qJCIiorzpavLBx40bhwEDBuTbp1q1arC1tUVsbKxae0ZGBl69egVbW9tc17O1tUVaWhri4uLU9tbExMTkWOf69evw8fHBkCFDMH369ALrVigUUCgUBfYjIiKi90ejocbKygpWVlYF9vPy8kJcXBwiIiLQqFEjAMCxY8egVCrh6emZ6zqNGjWCnp4ejh49im7dugEAoqKi8OjRI3h5ean6Xbt2Da1bt4a/vz++/vrrUhgVERERaYJWzH4CgLZt2yImJgYhISFIT0/HwIED4eHhgc2bNwMAnjx5Ah8fH6xfvx5NmjQBAPzf//0ffv31V6xbtw6mpqYYMWIEgKxzZ4CsQ06tW7eGr68vFi1apHosuVxeqLCVjbOfiIiIiqc0v0M1uqemKDZt2oThw4fDx8cHOjo66NatG5YtW6Zanp6ejqioKCQlJanavv32W1Xf1NRU+Pr6YtWqVarlO3fuxPPnz7Fx40Zs3LhR1e7k5IQHDx68l3ERERFR6dCaPTXlGffUEBERFc+/7jo1RERERAVhqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKghIiIiSdCaUPPq1Sv4+fnB1NQU5ubmGDx4MBITE/NdJyUlBQEBAbC0tESFChXQrVs3xMTE5Nr35cuXqFKlCmQyGeLi4spgBERERFSWtCbU+Pn54dq1awgNDcX+/ftx6tQpDBkyJN91xowZg3379mHHjh04efIknj59iq5du+bad/DgwXB3dy+L0omIiOg9kAkhhKaLKMiNGzdQp04dhIeHw8PDAwBw6NAhtGvXDn/99Rfs7e1zrBMfHw8rKyts3rwZ3bt3BwDcvHkTtWvXRlhYGJo2barqu3r1amzbtg0zZsyAj48PXr9+DXNz80LXl5CQADMzM8THx8PU1LRkgyUiIvoXKc3vUK3YUxMWFgZzc3NVoAGANm3aQEdHB+fPn891nYiICKSnp6NNmzaqtlq1asHR0RFhYWGqtuvXr2POnDlYv349dHQK93SkpqYiISFB7UZERESapRWhJjo6GtbW1mpturq6sLCwQHR0dJ7r6Ovr59jjYmNjo1onNTUVffr0waJFi+Do6FjoeoKCgmBmZqa6OTg4FG1AREREVOo0GmomT54MmUyW7+3mzZtl9vhTpkxB7dq10bdv3yKvFx8fr7o9fvy4jCokIiKiwtLV5IOPGzcOAwYMyLdPtWrVYGtri9jYWLX2jIwMvHr1Cra2trmuZ2tri7S0NMTFxantrYmJiVGtc+zYMVy5cgU7d+4EAGSfXlSpUiVMmzYNs2fPznXbCoUCCoWiMEMkIiKi90SjocbKygpWVlYF9vPy8kJcXBwiIiLQqFEjAFmBRKlUwtPTM9d1GjVqBD09PRw9ehTdunUDAERFReHRo0fw8vICAOzatQvJycmqdcLDwzFo0CCcPn0a1atXL+nwiIiI6D3SaKgprNq1a+OTTz7BF198gZCQEKSnp2P48OHo3bu3aubTkydP4OPjg/Xr16NJkyYwMzPD4MGDMXbsWFhYWMDU1BQjRoyAl5eXaubTP4PLixcvVI9XlNlPREREpHlaEWoAYNOmTRg+fDh8fHygo6ODbt26YdmyZarl6enpiIqKQlJSkqrt22+/VfVNTU2Fr68vVq1apYnyiYiIqIxpxXVqyjtep4aIiKh4/nXXqSEiIiIqCEMNERERSQJDDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQw0RERFJgq6mC5ACIQQAICEhQcOVEBERaZfs787s79KSYKgpBS9fvgQAODg4aLgSIiIi7fTy5UuYmZmVaBsMNaXAwsICAPDo0aMSvyCalpCQAAcHBzx+/BimpqaaLqfEOJ7yS0pjATie8kxKYwGkN574+Hg4OjqqvktLgqGmFOjoZJ2aZGZmJok3GACYmppKZiwAx1OeSWksAMdTnklpLID0xpP9XVqibZRCHUREREQax1BDREREksBQUwoUCgVmzpwJhUKh6VJKTEpjATie8kxKYwE4nvJMSmMBOJ78yERpzKEiIiIi0jDuqSEiIiJJYKghIiIiSWCoISIiIklgqCEiIiJJYKgpgVOnTqFjx46wt7eHTCbD3r17NV1SsQUFBaFx48YwMTGBtbU1OnfujKioKE2XVWyrV6+Gu7u76uJUXl5eOHjwoKbLKhXz58+HTCbD6NGjNV1KscyaNQsymUztVqtWLU2XVWxPnjxB3759YWlpCUNDQ7i5ueHChQuaLqtYqlatmuO1kclkCAgI0HRpxZKZmYnAwEA4OzvD0NAQ1atXx9y5c0vlbwxpwps3bzB69Gg4OTnB0NAQzZo1Q3h4uKbLKpSCvi+FEJgxYwbs7OxgaGiINm3a4Pbt20V+HIaaEnj79i3q16+PlStXarqUEjt58iQCAgJw7tw5hIaGIj09HR9//DHevn2r6dKKpUqVKpg/fz4iIiJw4cIFtG7dGp06dcK1a9c0XVqJhIeHY82aNXB3d9d0KSVSt25dPHv2THU7c+aMpksqltevX6N58+bQ09PDwYMHcf36dSxevBgVK1bUdGnFEh4erva6hIaGAgB69Oih4cqKZ8GCBVi9ejVWrFiBGzduYMGCBVi4cCGWL1+u6dKK5fPPP0doaCg2bNiAK1eu4OOPP0abNm3w5MkTTZdWoIK+LxcuXIhly5YhJCQE58+fh7GxMXx9fZGSklK0BxJUKgCIPXv2aLqMUhMbGysAiJMnT2q6lFJTsWJF8d1332m6jGJ78+aNcHV1FaGhocLb21uMGjVK0yUVy8yZM0X9+vU1XUapmDRpkmjRooWmyygzo0aNEtWrVxdKpVLTpRRL+/btxaBBg9TaunbtKvz8/DRUUfElJSUJuVwu9u/fr9besGFDMW3aNA1VVTz//L5UKpXC1tZWLFq0SNUWFxcnFAqF2LJlS5G2zT01lKv4+HgAKJU/MKZpmZmZ2Lp1K96+fQsvLy9Nl1NsAQEBaN++Pdq0aaPpUkrs9u3bsLe3R7Vq1eDn54dHjx5puqRi+eWXX+Dh4YEePXrA2toaH3zwAf773/9quqxSkZaWho0bN2LQoEGQyWSaLqdYmjVrhqNHj+LWrVsAgMuXL+PMmTNo27athisruoyMDGRmZsLAwECt3dDQUGv3dGa7f/8+oqOj1X62mZmZwdPTE2FhYUXaFv+gJeWgVCoxevRoNG/eHPXq1dN0OcV25coVeHl5ISUlBRUqVMCePXtQp04dTZdVLFu3bsXFixe15vh5fjw9PbFu3TrUrFkTz549w+zZs9GyZUtcvXoVJiYmmi6vSO7du4fVq1dj7NixmDp1KsLDwzFy5Ejo6+vD399f0+WVyN69exEXF4cBAwZoupRimzx5MhISElCrVi3I5XJkZmbi66+/hp+fn6ZLKzITExN4eXlh7ty5qF27NmxsbLBlyxaEhYXBxcVF0+WVSHR0NADAxsZGrd3Gxka1rLAYaiiHgIAAXL16VevTf82aNREZGYn4+Hjs3LkT/v7+OHnypNYFm8ePH2PUqFEIDQ3N8VuaNnr3t2R3d3d4enrCyckJ27dvx+DBgzVYWdEplUp4eHhg3rx5AIAPPvgAV69eRUhIiNaHmu+//x5t27aFvb29pksptu3bt2PTpk3YvHkz6tati8jISIwePRr29vZa+fps2LABgwYNQuXKlSGXy9GwYUP06dMHERERmi6t3ODhJ1IzfPhw7N+/H8ePH0eVKlU0XU6J6Ovrw8XFBY0aNUJQUBDq16+P4OBgTZdVZBEREYiNjUXDhg2hq6sLXV1dnDx5EsuWLYOuri4yMzM1XWKJmJubo0aNGrhz546mSykyOzu7HCG5du3aWns4LdvDhw9x5MgRfP7555oupUQmTJiAyZMno3fv3nBzc0O/fv0wZswYBAUFabq0YqlevTpOnjyJxMREPH78GH/88QfS09NRrVo1TZdWIra2tgCAmJgYtfaYmBjVssJiqCEAWdPphg8fjj179uDYsWNwdnbWdEmlTqlUIjU1VdNlFJmPjw+uXLmCyMhI1c3DwwN+fn6IjIyEXC7XdIklkpiYiLt378LOzk7TpRRZ8+bNc1z64NatW3ByctJQRaXjxx9/hLW1Ndq3b6/pUkokKSkJOjrqX3NyuRxKpVJDFZUOY2Nj2NnZ4fXr1zh8+DA6deqk6ZJKxNnZGba2tjh69KiqLSEhAefPny/yeZA8/FQCiYmJar9d3r9/H5GRkbCwsICjo6MGKyu6gIAAbN68GT///DNMTExUxzHNzMxgaGio4eqKbsqUKWjbti0cHR3x5s0bbN68GSdOnMDhw4c1XVqRmZiY5Di3ydjYGJaWllp5ztP48ePRsWNHODk54enTp5g5cybkcjn69Omj6dKKbMyYMWjWrBnmzZuHnj174o8//sDatWuxdu1aTZdWbEqlEj/++CP8/f2hq6vdXxEdO3bE119/DUdHR9StWxeXLl3CkiVLMGjQIE2XViyHDx+GEAI1a9bEnTt3MGHCBNSqVQsDBw7UdGkFKuj7cvTo0fjqq6/g6uoKZ2dnBAYGwt7eHp07dy7aA5XOBK1/p+PHjwsAOW7+/v6aLq3IchsHAPHjjz9qurRiGTRokHBychL6+vrCyspK+Pj4iN9++03TZZUabZ7S3atXL2FnZyf09fVF5cqVRa9evcSdO3c0XVax7du3T9SrV08oFApRq1YtsXbtWk2XVCKHDx8WAERUVJSmSymxhIQEMWrUKOHo6CgMDAxEtWrVxLRp00RqaqqmSyuWbdu2iWrVqgl9fX1ha2srAgICRFxcnKbLKpSCvi+VSqUIDAwUNjY2QqFQCB8fn2K9B2VCaOmlFYmIiIjewXNqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiHJx4sQJyGQyxMXFaboUIiokhhoiIiKSBIYaIiIikgSGGiIql5RKJYKCguDs7AxDQ0PUr18fO3fuBPD3oaEDBw7A3d0dBgYGaNq0Ka5evaq2jV27dqFu3bpQKBSoWrUqFi9erLY8NTUVkyZNgoODAxQKBVxcXPD999+r9YmIiICHhweMjIzQrFkzREVFle3AiajYGGqIqFwKCgrC+vXrERISgmvXrmHMmDHo27cvTp48qeozYcIELF68GOHh4bCyskLHjh2Rnp4OICuM9OzZE71798aVK1cwa9YsBAYGYt26dar1+/fvjy1btmDZsmW4ceMG1qxZgwoVKqjVMW3aNCxevBgXLlyArq4uBg0a9F7GT0RFx7/STUTlTmpqKiwsLHDkyBF4eXmp2j///HMkJSVhyJAhaNWqFbZu3YpevXoBAF69eoUqVapg3bp16NmzJ/z8/PD8+XP89ttvqvUnTpyIAwcO4Nq1a7h16xZq1qyJ0NBQtGnTJkcNJ06cQKtWrXDkyBH4+PgAAH799Ve0b98eycnJMDAwKONngYiKintqiKjcuXPnDpKSkvCf//wHFSpUUN3Wr1+Pu3fvqvq9G3gsLCxQs2ZN3LhxAwBw48YNNG/eXG27zZs3x+3bt5GZmYnIyEjI5XJ4e3vnW4u7u7vq/3Z2dgCA2NjYEo+RiEqfrqYLICL6p8TERADAgQMHULlyZbVlCoVCLdgUl6GhYaH66enpqf4vk8kAZJ3vQ0TlD/fUEFG5U6dOHSgUCjx69AguLi5qNwcHB1W/c+fOqf7/+vVr3Lp1C7Vr1wYA1K5dG7///rvadn///XfUqFEDcrkcbm5uUCqVaufoEJF2454aIip3TExMMH78eIwZMwZKpRItWrRAfHw8fv/9d5iamsLJyQkAMGfOHFhaWsLGxgbTpk1DpUqV0LlzZwDAuHHj0LhxY8ydOxe9evVCWFgYVqxYgVWrVgEAqlatCn9/fwwaNAjLli1D/fr18fDhQ8TGxqJnz56aGjoRlQBDDRGVS3PnzoWVlRWCgoJw7949mJubo2HDhpg6darq8M/8+fMxatQo3L59Gw0aNMC+ffugr68PAGjYsCG2b9+OGTNmYO7cubCzs8OcOXMwYMAA1WOsXr0aU6dOxbBhw/Dy5Us4Ojpi6tSpmhguEZUCzn4iIq2TPTPp9evXMDc313Q5RFRO8JwaIiIikgSGGiIiIpIEHn4iIiIiSeCeGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKShP8HfkdBsXySsqUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "    /* background: */\n",
       "    progress::-webkit-progress-bar {background-color: #CDCDCD; width: 100%;}\n",
       "    progress {background-color: #CDCDCD;}\n",
       "\n",
       "    /* value: */\n",
       "    progress::-webkit-progress-value {background-color: #00BFFF  !important;}\n",
       "    progress::-moz-progress-bar {background-color: #00BFFF  !important;}\n",
       "    progress {color: #00BFFF ;}\n",
       "\n",
       "    /* optional */\n",
       "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
       "        background: #000000;\n",
       "    }\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      <progress value='8' class='' max='30' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      26.67% [8/30 51:02<2:20:21]\n",
       "      <br>\n",
       "      ▌                   3.00% [3/100] [train_loss=4.33695,train_acc=0.00000,lr=0.00000]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.fit(\n",
    "    train_data = dl_train,\n",
    "    val_data= dl_val,\n",
    "    ckpt_path='ctc_crnn.pt',\n",
    "    epochs=30,\n",
    "    patience=10,\n",
    "    monitor=\"val_acc\", \n",
    "    mode=\"max\",\n",
    "    plot = True,\n",
    "    wandb = False,\n",
    "    callbacks=[visdis],\n",
    "    cpu=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 四，评估模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-18T12:48:58.168479Z",
     "start_time": "2019-06-18T12:48:57.536996Z"
    },
    "code_folding": [
     3
    ]
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d795fa217a30446e9027ab392b6e5990",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def display_fn(model):\n",
    "    model.eval()\n",
    "    right = True\n",
    "    while right:\n",
    "        image, target, input_length, label_length = ds_test[0]\n",
    "        output = model(image.unsqueeze(0))\n",
    "        output_argmax = output.detach().permute(1, 0, 2).argmax(dim=-1)\n",
    "        right = (decode_target(target) == decode(output_argmax[0]))\n",
    "        print('gt:', decode_target(target),' ','pred:', decode(output_argmax[0]))\n",
    "    display(to_pil_image(image))\n",
    "    \n",
    "    \n",
    "from torchkeras.kerascallbacks import VisDisplay\n",
    "visdis = VisDisplay(display_fn,model)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 五，使用模型\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(model,image):\n",
    "    model.cuda()\n",
    "    tensor = to_tensor(image)\n",
    "    output = model(tensor.unsqueeze(0).cuda())\n",
    "    output_argmax = output.detach().permute(1, 0, 2).argmax(dim=-1)\n",
    "    preds = decode(output_argmax[0])\n",
    "    return preds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.load_ckpt('ctc_crnn.pt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMAAAABACAIAAADDDu+IAAAibElEQVR4nO192ZMcx3nn92Vm3dU9PWfP4BgMAYL3IYGHREIiKcu2wit7Y63dcOjJz/6T/Kwnb8SGHhyS15JD5iGBIkEKBAmCAAiCJM65+6ysMzO/fajunsZgZjCY6YG8En+BABoz1VVZWb/87i8LO5mCP1Gc+cd/Kj+c/tk//3FH8v8v7juH4qENhYiSONZJgoie73PPO7hrFVF0/V9+Xj15AgCO/fQnRRRZYXhwl/uvjCKKyg8POgPJ4vLX//vn5bcqj51IFpe9ufq9hz08AgFR0mx+/t7vx5Wee+GUMzlp+QF33QO62szrp1fePjPz+mlvrv7nzJ5kcbmcB5h7YA4BgKgEVhge/vFfi0owfNrygxWG+NBUGBmzduXzW//267zVdKpB9cTx2Ve+60xMcOdAOLTnlfcng5I91//l5wDgzdWP/fQnDzQVw+QbXoTDP6+ePPEwJRBYhoQyWSvKorTVTU2mjvzgdRxnzLFHfrU/W94MY+XtM+WHmddPP+h3rTCEOTj205/A0GQOk3Ll7TPVkyfY6Ea7IwiwDdBG6AAAgjaqI+WV6+tnL1CeIz6kUfy5oRQex376k73pcSsMyz/DP9xEyocngXRHwZtKa40ISEjKKBkp2dFpJqoVAHpoI/lTBckEiJIkwcADRDcI/Ln6JhGyfwwbl/DQCEREWdJdG1+kxQQZAkApdAwS4TfU2Rd0koFZ16lUMpJnrtP8oevry8cWHsH6jBX4YqSq/F699tAkEBW8kyVXC56CGfzsG7mzbyCYLGt+cN4Of7P2adhtVPSnFyzfv/3p1Xy+Pv2XP+Rh0F+to8EmYfYQ40AqJ1WQyREeluE1KhChlBDHgABhSJ7/xxrIJtdSZ5mWUXJrUV6/uvS1UmZdJQkZZrqSk+mo3Pvya0IuwkAEBzXmh+iFAQAgAgL2BA8Zo5PcJLkpFAo+qmsYY7IsI60pz10hsChQCK00uC4TYm8en2p3zJtvKmL4g9fFJBeeM6rR7h7D/jPOIbNE3myunD2bXLqcd5JMeqQNYM8h0YhZki395p3w+p36D1/7kyEQAVHJIgIgo7PlRnT1jj05zi1vv+qMqIikknEhZbvVWl1anA3HGo1Wbcxnkey4vjZm/JF5OwhE6LMwJGHtfL4iioBASQmA6ZdfN7+8k2WKJzjzxkvOVE34gXVgUdAtBzPsPy/89H9Ska+fO986fzFvtkFpAgZYinYCAEAsZKxkzKtjSsYHN7CHQiAC3U11lGqZ9tRxKYaMSRvr7ds3/Oa8a9nc2p8QIsgazaW3fpeuNfIsA63v5IUjeFsVKLjWhhy3c/7TSc8Ze+4Z/tzTfGyMbS/2Bsu9+vijrc8up6vrKk5Ukonr1+J/XQmeffzYK99Bz6WHaMOtvH0GEHSejz33lLx1y+R5cnspb0ekDAAveUOlvUM9o4cACylVHKs4Ef6B5I7uQ6ARxXOpaEft9y6bNINSgRESIABplbZv3mFXbsyPV9D29vM8iEh1o2RxOb61iAzLdZgAAFLfiuwWZDR3O9JM+mHt208D8i3F3mC5KymbFy5y11VJAoAAWHQ7lES276YnTwrHY87D02Uzr59eeefdyRe/FS0trp07r1pdLSUYAsDyHku7oD/D5XhBRXLpzd8dCUN3evIgOLSTPTuYx2RxecCkPYAItJQmbpmeBCofJwEAaW3WW+z6qlnOMdmHs0BUyKiQsYoT7As5ABpiDwAQIUuKLIq7WRQZVeBW7omKk3R1/cb/+ddkZTVZWaVCKRlDqXSBEIAbyJfXG+cv6izbfwi0iKLyz86HWWHoz83O/+TvuOsUzW70xfX49mLe7pLWgAhY2pXUv3fs0QdASZksLS++fUan6Ui9sR62JdAwewbBx31AE2jDTG99AGH/ETOV8stt9kHXFPme75AA8lZ75Q8fmizb+CkOmEoABAiEAAxMnijZVVFk1D15QASdJstvv5u12zpJuesC4z3bojc2JK2VjLWUOk326SHvfomaQhmlik6ndenz7pUvwBjs2csIQARoGDeMbYxnII+ICimLJFNxPFp/vsROKmw/mZRhoIwojlTWBYBS8PQfLAISKZMWt2PzZZhNMLiPYbstiAoZ6yhVcYzlckTcrJ76/yWtW5cve4fq9vg4ig0tppNEyTi6cTNdXdNxyjwXacDCoVEDktZaSiVjlaXc3qMW22QXl9G5rW9OqaLTbl24tP7h+bzTUTKmHvXRIKHn2EEIY9OMkaWXjIxUJHPpACEglFYBM8XeBnlf7ESg0VREEGG3yz44a9LY9Jx4AGQECGQAwKCSVtR1Oj7POAS0V2estKr6ZiQwJrjnAoE2mU7S/lwileGTKFVRTEqh7ZRX1GmarTeW33k3un6z6EoAAERCsILACjwBpGSsoqQo2UkUL6+sf3rh8NQEOs6eTbf7LlHSWkVR0ek0P720/sFHRburiwIAARGIuOPyqktTtZlTL4QzdURm4ptm7We3/5NjhD2jGhAMqa5UkdRxPPIyrG0JtGUydl+gUpUgci78AJCZPNNxTAAaSe8vKF1EsYpjFccAgMh4JbArtdr8SVF3m1cuKRlTN847cdGNmG2XLO5TGXSSaCmz9cby27/vluwxpvy1CHxvdnruL14Xqkh/8593DBVxAohG6SLqqm5XpalVre157PdZoghktLxxc/m3v0+WVpRMTV4AAHMsEQZWWEHfrj33ZOX4cbtSsYMAOaeM3f7iZeZeA6ZAD05ERae7/sE5rz7N/X2HS+7GThJoNLxBpEpFv/wyLK+DTAGAAQZTk+7h2fVPLqqYABEKhVKCjCAMwHrgQB8RFZ3u6gcfmyRHQEQezNTrb3zPm6uTMdUnT6ZLK+0Ln6+8+x4ZY/J8UyzR5Pna2Q/aFy5nra6Kk5I9gASIdiWc+8s3woV5oY1Wmv/yVxQnQARGG6koU1SoPbNnV0sUwRRKRbGKYlMUAIi2ZVUrUy+fqjz6iAgCEfhWECDn5UQYPT5+6q+iOyg6t5SMyGyciQ7ChN7ZCxsVKAgoCMgPSkfGYVDxPMd3uWf3bFtAc2vRfHnTmD3ZeQRKxqYrlUwAATizqlV3ZsKdmXTrU/7cofCRR1QiReADEdoWAWlAg6U8BJ2m6WpLLq/l3cgYU8Y5AcgJPG9+1p2ZEkFAYQBBCG4IRKC1TlOTFelyQ2dqP6GHLesl7oIBe7wWHJsnbQCQiBjDiWefmvz2s5VHjvlzs3a12mMPAABwz3Mmp+beeNWqVYen0qpWpl76NnfdkScfHwaBlIyVjFWclA4nIvlcEfadI0AqiqiwOtqjvNgDg5SUSkZFIhEJGOMVD6s2990h2cAmXniJmI3CYYwBMs2N2agCQEKk0iwD4IHnTk2NLRweO3509vSrIggAABCtwLdCX7ieTjPShrsOd13uHXAwmmHebMuvb5QsQSBu2yLwROCzbSLpwvdE4IvQH/LIDnCABx+JJiq63dWz53SSYul1ITUsVg09Ebjc9XSWG2B5FmdJVw9Cqg90BWOaFy8BAwJgDNzJ8annn2N92xYRhO9mkUG/zmQEANyx7MC3Qp8JQT1vtxdVAALmOPXXXgmOzFlhKMKA+355F2iJ6tOPtb/8EgCsSujN1o/+3Y8OuO4RSSklpYpjk2fAOFC5AnFHUpDwfcsPROgX3ahcpEWns/bhOa8+NXIb6KFIoCguhVC51jUgA7Cmpyaee45bAomAjEo7RWdFdzqlnfhAQMbGn36yrBJBZLYfWL4v3IG7gaZQqCKm2mVEh3Ex8fgT1aNHkfWEPw4FFyyGbhiG9bpTr/Ogzw9ErVTz0hXknLuuM16b//v/5s7WRRjAAYJQCBEGPPTRcYCIAADZ8FPbIg5JIDxv6uVTViXsxSAIlJQ6kireb+DqXjykygosAzMAyJioVNjUpDs1KXyfuy4QIQIYk1y6Jj/6FNotVA/KIURgSAwJABgA22QwIhgGioMC0ETAbdvyfcsPmBCD5YhEZQRAxxl88knR6cKQdaO6Ubq8kq2smjy3x8fGnnrcm6tbB8seAAJkLDhyqPbUE4xzIkIgIEP9iqrt4pA88EXgizAYmoXNvNllBPy+OHACFVKqWCoZYz8Vb01Ojj/9DParaggREKlQSacbf/JJfvmK0eoB1glR0ZVr719QaQKM8SCAsCp8b6AJjVZFHBdxrPIcEQwjxVAxNHdzbMMgQlrJzV1iPorU4mLzrXd0losgcKcnx55c0ElXZXLP0/JAwIGKJRjQ+v6pgs3m/cYNjypJBQ+BQCqK1s7+QSVJv8qAWZ7HPc8KAh4GrBJw1ymJZYq8GactmVK+dZZqSxBAEck0lkrGAMQda+LxBSpTjFpDp63uLLXOX1y78JkhhYxZvptP19TEGLOsAckINtJJZNvjLzwPlUpf/pPqdLI334rXGzqOueuGR4+uvvXO0vu/kjduZY2mHk6ejBYIZIy8ead18bJRqpfzGjKAtotDaimVlFrGG+YObeQER5ukOmAjmkjJWEmppMS+lQqESACM+Yfq488+vbTaoCxDQAOgCBShBtx9YQfGMaYxTzpl+sIkSePsOe57RikwGm7cbJ95f6UlCylNlnPb9urTR146VX3kGDBONEhUmEE5BADDcoT9OS+ieEWmSqbAuU7StQ8/AqZgrR1/9e+155+vf+cFdPcejN755npGtIxNlgPnQKCLQiWxSmJuO/XXTy9vGYdk2PnsMzClVdg71bBzMqokFTwUL6yXJC4dCCsMHb/CPQ8ZE2Vtl2dT1CtgMXlWSKmkFJ6HYndjI7Kufo5l6M9QEcnO9Zvxz38RPDLvT060zp7LG60iz3rhJsErx0+MP3rCCSvQs6BJuI4VeFbg54UCMirNGmcv+LN1FFrFiepG8epaIlOFiIaU7KstMiZrFIfWTZpBtXow1d09I1qEQRFJUxSAAEXR/eJqcGR2/NlnvLnZreOQhqpPPdX+4jrgcGbIDAY5wrbdgyWQiqWKYy1lySFkzJ4cq734dBnRQs5F6PPQEx3HZAUiktbdz6/5h+rOeA0tsZuHopI4OXSo+MOFskaNiIpOpGRStLstADJa5xkBABGzLDFWs6cmrUoFOR/IDGNo7NHj3atf5c02EGqZmG5bRTG6tPzuL+LrWdFJdZz06pjK7yACARaauomSUmRVZo++NxIAwJA9PhYsHJV3FgkIEanIsdmiVouKQlQq9xg6AAA8CHgQCj80uIK9YEY/OT/qJNXBEqiI4tWz7xdxv6SSoQh9EXjcc4kIGQsOzU0+9dTyyu9MmgNDk+daSh1JUyju4m4Sq7mm5YufqUHMBwAQSOueqMBevQNalqhWJp5/ZvyJR4GL4WlnDDtffNmLc5IBgDxJ09XVtAudS81keRUUK4PWgzq1Ul1q0tHiLffiZWdqAh37QLQYw7zZjr6+gZyB1mWki3HOOQe2k/1KCJngBIBD1s8AIwxfHbgEKmRaxBKR9Zt4NmxA5MIKAivwue0UFGO/WPoBGsWIlIxNlKo4QRiaqcEp+suPoxh/7Impbz1t18buVo6ok1RFsZKSyCACAOVZfOedd4G0jlMqkIyhPLfHa8gtRCxk1As/Gp1LWchIZ5mojF6LISApVep0k2XIeOnFm9q4CSs7X40AgOmNiqjSqjsAHLQRXeYlt46pD6qCqHcQEpDum0Hcc5HfZ3hKJjpKTDfB0tJCMEgECMSwp27KaUNGxGioangTSteGem0jKpFKyrKvA4UweS4C3x4fq3/ve+sffmJIlcVZyLkVBJYfcMc5CBuIgFBwHvq84vNImrwAREBedLoqTqhX97Q1EMjWlPbqospiyt27tg+AAyRQmiRpkqRxAlBWIA4KdgYVXAqk1ElcqBwYL/UCad2+ctU7NGuP11DcxwxSMl5//5MilkCAiCLwCouB51JsPIurOCFTqDRFAE26df2adW16ZnIcLWtI3RD3XMfzrMA3heqZFAbAaJOmAEBZJsLAq08v/MPfCz9Ax775f5sQp0CEjHmH58aeeQLtg9FfBMh4eOTw+JNPrKw0NBSlTKJcUa5A6+2+p2SiZaKiGA0NFvHBCKCDJJCS8vOPP7YGbgsCGZNHnWRpRWcFAAEZc/2GOneeNCNEIEIAkxdJsxGvLgWdw+7ENLKdRqhknMdtJWNAQkRnYrz24nNrsTxcndIrTVF11j44Rw0yWaZVUchERbHJc/D84aVLhoLHjreufZU12/2Rgs5yAADk9tSEM15b+F9/683WeeAXWe7M1E0c6zhGzkUQ8DDgjksH1p+BBEi4EXCAwdi3kScIRZosffBx3u2CMaXlj1sUaI4GB0qgeGy90ZGx2IgAUbrWvv3vb1qeS0jQKwyVLM0JWD+tiaD02tVP7COTztgEu9vg3QQqXVMkMEAceTWoHj08e+iQSTLnW3YhpQj9xd+8k7faKkmQypDuPfPOsHXtqyKOkWHZmwcAzLZ1kVuTU86RhYUfv+FOj4swACJ3ojb9/JO379zWUu6c0hwJEDFvtqLrN5jgGqAMhrCyC2N79VXESZp1cxkzo01WMMcWfiiCQPj+yFXtARJIEFkGLTPkaRrSnShpR2kpbQD68XaEfi4UiHSW85ahdm6U4ra73T3rKNYy1pEsf08IhpPne34QkO8DAA88Jlje7iy9+TskADAI5h7PDnWaatktYmm07heqoyW4X5uwDk0f+ts33Jlpyx+UbQyqUDbSaCOYrG1ARPZ4LVyYj27cIgLEnql4H3GCAGSRIZ2kAGiKwhqrTr78AndHnIqHA05loEE+NL+loVqmcwblE2USFHHI5EVCRpwR39nqU3Hc+v1HqgzYlwGCMOC+N6AI8l5beC9b0m+WuhvEPZeFgQiD0s0pRyEqweH5Qws//pE3PTFgDxFl6421c+dNnvWGj3igQohx0bsFx+ndFpVbmuz4LQLLGCi1MAC3HRGElu/3SlNGO8KRn7EExjFISVISDTc2DH4NQBthuR6G8pk6y1SjoZZXqN3azlosZBynUdm3i4hObax+6tR2i4yg5w1u+iURMNuZeu45Z3oG2KDvAoQX2K+86s7MWEEwOFR1Zbq8lq+uqyRhQlhhYAU+cw/EBbsbgwYeAESdF4WMtZS0VdkCxhJlxDsNLjgyzl3XqlbsoCxsGuU4yyzsjpUl+0Ec87Pv8agNMKQ0BmSi/mra8MsG0gEBwWjdOH9R/tt/mJt3gMy9ckPFcZHIIpZUZkKBOX5g+YHYbusMBEaw5W5EwvWswLcDHznvPSkiYTvkejC8ZInyRrvxwQWdpgjIgIX1+sTTTzHrYFywe0FliIyQi+zGjWK90Wf8EBB0ksDZ902ng8i46wLnohpOvnxKeKPUX0UUqa4884//dJcNtPL2u5XHTiSwbIVhuT3ng560/FB+8a7ljj1nYAi90mMYhDP60WQENHmRp40m2XYzCrRGy960eookXv7grOpG5YWsIHC8ymYjUSsdRVpKk6UIyAn4TjNIAFSmVxkQ4d12BlERyWRlOV1b1EmGAC7jVbfihD5z3S3zCaPFcLmJ0dqZn7cmJ8AQbE47o47jRhQVUQwAgAwZWkEgAp8H/v7HWfKmfNCNcx+f/tk/30WgiVPPL79zZvD5gZg02I1g4tTz3tysxVC9/B1YbEC3X+2AvTj0kPncL5hDICoDXWX6AhHAIOvmcSgTu5taloNDqy1N02htPVpd05Esz+oINv7sY3dNJgIZk9y+3f3kglEaEDVyc8983/WFPogYAYPBwQR5lCS3l5bfPqOzDJAYY17ojo2Jh6K/AKC/+giJiCEKxjhjeE94GWWkoyiPkmI4V7NvQ2UTb8of1l+7e4/EkiX1105vOq5kUnnAdmQalJgUUdS5es2bm62/8T2dZgX2h276beo925MRAoABIgRjgJjj2rbDiUyhiiw3wAyQUnnr/FVv7pA1XhneCAERv/riGte8J+aQtFHq4/Nx6AmtLM6E5wNnBqiIkizOdJIy22ZhwCohszcLs/KUZbAWsLSLkeuyFBoAQckoubN8/ZdvqfWmjhNk3A497+RR/t0X2QGlwDajF0pGAETUSpkvrpqjc6Y2hnxofwgElWbm3ffzZplsQUCwwgCqrnhw8TNMGrinAqQkA2xy48sWEw/qRRTdyyQYIhPcw6eVt8+UNAIAFSfZ2joPgqIbAQCQKRuyHNcpv0JBoBEBC6E1lW09llU7tjCmVOPji2uGKFeIzKRF3FyMV5bcI5NWbQz76UMiOn7i5M2L10qZRQBpFH/11W1v5RfiyGz9+HGr0VCPPmoYSxstlRUAgJxVjs9XTx5Da4vAEhKxXsFomcxggEn/2VHR6a689VaxdkclKSIyRGd2tvLqaZyYpIPZ5HoTeukYKk1WhDxvtrtOpzuuFFgbIlAnWb66fqeRZlFM/RSOqIQzL77Ed7EvxybGwNCjV93Im5tdfufM4R//yArD4S3rt44DbckkuJtMMMSn8gKNc58AgFUJkTFDUCwu9cwKY3SSWgwcx5o/dhhf/z45HgGVu0z1fAvbth0HZGzlmr1/TvdKu8DofO29962xoPLYifLMAOB6nvI81/eKvt2tjIF2p+h27U4nu3KNewF9esVYQnUjowq0LKdW8Wcm7IkauyuPsfGELFAIpte4jEZxIDRZLFUcd7/+Ol5Z07EsWyJEtSJm55ypSe4+pP2B+sq1DD4R2bZxA/JC4GKDPWmSrTfvvPOHqNnMy7Azkl34gTXn2VNiyBu4lygD3Pt8AUB1oyKSt3/5KysMu59/MXHqueFj7hNIHDCpvPAwmTZdT3WjymMnVDcChLGnn1g7+wfShoiIjMkyMoYA56Zq1munvYVj5G6xIBCxsFvk+2BbmKRluYJO05xg6T/PMNetnHxEuM7doZyeN9cLIxnKu0kOgC1ZEpQAiYjbdriwUH38JBNbsAcAEClH3ucyEBAhFDIxK6tf/fa3cGtJyxSQWWFghUG4MD/zygvMeWi7S5XLTPdu0hByXnvq8XD+MPQDVzpN80bj1n/8Rt64paIIiIAIjIEir1yfie8sZtAaPuMmogxQMqbEQMOULx4pVc29FYwPEIkeJhNsxacSGMd3fvUbnud5kfejhowJZlfDVW4f0ia/dWdLuxMRdBzrtXXMUgZk+slxlaamub5+6ZI1ORbMziJDIBKBL3zfCvyiKzcY1TPTe5GDMmDLhLBrY259xp6YYNtkPcskh+mZaeVfpOJ45dz54usvdSdhhFYl8I8envvB95zJyV5w8mGBsJyNIeMtTfPVtbzTLWOyJk+Xf/9e9OXNolNuOQUEIFRuuZXu9BV+pUlLd51wmCgD7GDj7lDBuPdUxiY+9YCQ3rg1HXg3co1CbDjunIv67MQbpxufXd0h70hFka+uhYI3jabBjBGoOG5cvuxNT5quRMYIQMk4ODLXvfIFaV1SBcuKPYaA7K4YAmPBsSNjTzyKWymvoYFDL8UECABKpunKWr68Qp2UAVph4B85dORv/9qdnBzJBhc76JG7RoWgsyxZXSvSlLQBVERk8nz9wmfx6ppVqzEhdJKma2vJ8krR7pKh0s/lgWfP1Q5PT/PvfJfufuQPGqPZuYJx5LkwRMYb7jhZFsOeDEAE4oJNjNtjY5XXjubdbWcNEXSzdeuX/4EdqfO876YiqVzEbuujT/PFFewrfiVj//CskhUA6O+wKXVRIPQDjwRExDnTaZatNXSWb6O/IG+18kaL8gK0Lg9R3dbqu+8DABW9Taiqj53I11smyUaluLbTI3ePDVWapEvLTJEuCkIGAKrIweJjM1OV4wtUqGy9Ed24qWSKnCNHAGCubc1OHP2LN4KjR60whGC//Ws7sG30yVQFqFSq+loEenYFIqEVht7crDu77SNAhLzVnlhrrZx5D9oto0zpuhIRQ1576olwYZ5xRgCqz0JRCVU3IoC81W5dvKIW7+RxrOJEARNMWEFg1aqgdfPCJe7YW14YAVSagjbcsYssImMQIG+3ByMXlcAeq8Q3bmdrjRFO1JZ65N4JMVnWvvhZtrLMLEFlIwAZy/PtatWpVdcufta99IXOcmZZpHVpwIlKMPHCKe/ECatW2yKPNFKMnkBIhhtCMncvd7IN3TfmRgTC96vPPSGbq+sfXYBUsfL+jRGV0JudqTz6CLOsottNAEqtXC6O5bfOTLzwrcM/+gE2GskfPrxd6EAxhpZ/uD7+7GPM3po6vQH3d5daevtdFadozCC+iYAi9P3Z+uwPX9uNJ7x77FaPIJg807GMvvraUFMXCgCQmCmKvN258+578trXRSuCntwE7nsi8J352YmTJ60wPGj2wEGVc2wUcveMClZxTNUSQ6ny7cBsy52oFUfmzI3bYnmdlO7vKt07oNyKtdwZ7vYvf11/7dXbv/w1ADTPfbzwD/9DzNWdhXlnZU0B45x7szMi9PmOe6mWYq91+arRhnEBbLCpDoow8GZnjv73v3Fnpva/V/detrwlQMaDY/PVJ59M19/XeQGIBGC0Wf/oAiDoJAXVq2EiAOa4M99/Zezxk1Y15Lvsi9oftghy7yerquNYySiPuzTIgCIgkF31Zl78NtvN/jQEYNlHTpyYPDKHbFNCqvfvICpaf+3V5XfeLT/PvP4qMYRKBcMQPa959kNmC+ZYO7NngFJZUr+ov6y9ERV/9offHxV79tZNzBAtxizGWM8xQESm00xHUnclqLJUgRCM8B13erKycMyfnbGC0YufLYmxmUD76ppG0Gmy9uEHebs1XH3DAD3Xtz1f7KIehQAsy/YrlepYxQmcQbp+uKh36DVYs4d//NfDr8TaQ98uETDHmX3lpcqTj4WPHAuPHwuPL1SOHwtOLAQn5kfOngfrJkYAY9jiIv/yC9B6UNo5lOrt7dnOXTucqB56/RVncuIgaty2I4bY8iC4376hWwIBdZxomalIlv5k714JLcN3f1cExBzbOXzYvna96KZameHfikrowcZrsEQUbPIw99C3y13XmZg4/FdvqEiSMWU2l5DKPeR2eZKdsa9uYiwjYv1kBmxEvIBIeI7wPWdyrP7Ga8GxeREGByF7tiPGZjU5gq7p3hbQGwuEhwFVKiLYdUETAVp2uLBQLK4xFNoYIrBrVataQcaAaNiGuNee2FvfLvdc7rkwNbnL4x8Ue+wmJgDGYG6uOPlovrSKg/19+gxRCNy2a999afKZJ+2xKg9Gz54S2xFjM4H20zWtc60yrVI92DEdECyO7nhl5pUXmefvvvABGRdBMP7dFyvPPGm0JgDhebsRBqPfXHYU2NeouNCeD7ZnMSuDGAGAiDuOCAMrDIizaHrcOXncm5tlggMd1NuGtyPG5mz83u8TQadp88pK3pZ9L4zKbpLZiSp/8IIm5ji249iTEw82jP9KvBnGnkdFQCiEMzYWHj7kjI8D40DEOK8980Qwf0Rzzjy3p7ZoBO/53nbw2xBjswTa2yUxy3QcZ3dW46uf5u0W9frZEAgE402t5+8tvvwGu0TZXnjiEbc+o7McEYmIO3YpgYbbtHdjwu5ZPm13/ChCBQhFkevz55fOX0rWGqTSjQg0IObBDFUp93ZrAH2De4BC2LUxuzZ23yN3NmEPQj6NoCsDAU2WfN1otFdXim4Xertb9xKplueLzretYPQdSd+gxHB4Zof3fI/67Tk9jCRYSeS4WK3aRw/zrBi8vKpMBbjuDPzlxJYFQN9g/xgWKt5c3dvxPd8j3JhsAOxk97zy6MFRpKmSsogk6sHelb1/y764g3tn558zho2ekjo7aKVNVBuVfT0aAn2DPxbO/OM/lR9O/+yf/ygD+H8uTrz02+s0rgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<PIL.Image.Image image mode=RGB size=192x64>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'ABBD'"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generator = ImageCaptcha(width=width, height=height)\n",
    "image = generator.generate_image('ABBD')\n",
    "display(image)\n",
    "predict(model,image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 六，保存模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.save(model.net.state_dict(),'best.pt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**如果本项目对你有所帮助，想鼓励一下作者，记得给本项目加一颗星星star⭐️，并分享给你的朋友们喔😊!** \n",
    "\n",
    "如果在torchkeras的使用中遇到问题，可以在项目中提交issue。\n",
    "\n",
    "如果想要获得更快的反馈或者与其他torchkeras用户小伙伴进行交流，\n",
    "\n",
    "可以在公众号算法美食屋后台回复关键字：**加群**。\n",
    "\n",
    "![](https://tva1.sinaimg.cn/large/e6c9d24egy1h41m2zugguj20k00b9q46.jpg)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "3e594ae65a88453682c6b98253d9a46f": {
      "model_module": "@jupyter-widgets/output",
      "model_module_version": "1.0.0",
      "model_name": "OutputModel",
      "state": {
       "layout": "IPY_MODEL_6a8e4a2263184121b6ce6dd5619e7112"
      }
     },
     "4e3c790e04ed41dc9653642087a09bde": {
      "model_module": "@jupyter-widgets/output",
      "model_module_version": "1.0.0",
      "model_name": "OutputModel",
      "state": {
       "layout": "IPY_MODEL_5698b97586c1475e857f3f74dbe1a9f0",
       "outputs": [
        {
         "data": {
          "text/html": "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #800000; text-decoration-color: #800000\">╭─────────────────────────────── </span><span style=\"color: #800000; text-decoration-color: #800000; font-weight: bold\">Traceback </span><span style=\"color: #bf7f7f; text-decoration-color: #bf7f7f; font-weight: bold\">(most recent call last)</span><span style=\"color: #800000; text-decoration-color: #800000\"> ────────────────────────────────╮</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/lib/python3.8/dist-packages/torchkeras/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">kerascallbacks.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">243</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">__init__</span>              <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">240 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>display(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.out)                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">241 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> model <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">is</span> <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">not</span> <span style=\"color: #0000ff; text-decoration-color: #0000ff\">None</span>:                                                              <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">242 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">with</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.out:                                                                 <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>243 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.display_fn(model)                                                     <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">244 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">245 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">on_fit_start</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>,model: <span style=\"color: #808000; text-decoration-color: #808000\">'KerasModel'</span>):                                            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">246 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">pass</span>                                                                               <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">display_fn</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">6</span>                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 3 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>right = <span style=\"color: #0000ff; text-decoration-color: #0000ff\">True</span>                                                                            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 4 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">while</span> right:                                                                            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 5 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>image, target, input_length, label_length = ds_test[<span style=\"color: #0000ff; text-decoration-color: #0000ff\">0</span>]                              <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 6 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>output = model(image.unsqueeze(<span style=\"color: #0000ff; text-decoration-color: #0000ff\">0</span>).cuda())                                           <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 7 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>output_argmax = output.detach().permute(<span style=\"color: #0000ff; text-decoration-color: #0000ff\">1</span>, <span style=\"color: #0000ff; text-decoration-color: #0000ff\">0</span>, <span style=\"color: #0000ff; text-decoration-color: #0000ff\">2</span>).argmax(dim=-<span style=\"color: #0000ff; text-decoration-color: #0000ff\">1</span>)                     <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 8 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>                                                                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 9 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>right = (decode_target(target) == decode(output_argmax[<span style=\"color: #0000ff; text-decoration-color: #0000ff\">0</span>]))                         <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/lib/python3.8/dist-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">module.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">1501</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_call_impl</span>             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1498 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">not</span> (<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._backward_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._backward_pre_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._forward_hooks   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1499 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_backward_pre_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_backward_hooks                   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1500 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_forward_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_forward_pre_hooks):                   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>1501 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> forward_call(*args, **kwargs)                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1502 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># Do not call functions when jit is used</span>                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1503 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>full_backward_hooks, non_full_backward_hooks = [], []                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1504 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>backward_pre_hooks = []                                                           <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/lib/python3.8/dist-packages/torchkeras/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">kerasmodel.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">114</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">forward</span>                   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">111 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.from_scratch = <span style=\"color: #0000ff; text-decoration-color: #0000ff\">False</span>                                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">112 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">113 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">forward</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>, x):                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>114 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.net.forward(x)                                                         <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">115 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">116 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">fit</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>, train_data, val_data=<span style=\"color: #0000ff; text-decoration-color: #0000ff\">None</span>, epochs=<span style=\"color: #0000ff; text-decoration-color: #0000ff\">10</span>, ckpt_path=<span style=\"color: #808000; text-decoration-color: #808000\">'checkpoint.pt'</span>,         <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">117 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span>patience=<span style=\"color: #0000ff; text-decoration-color: #0000ff\">5</span>, monitor=<span style=\"color: #808000; text-decoration-color: #808000\">\"val_loss\"</span>, mode=<span style=\"color: #808000; text-decoration-color: #808000\">\"min\"</span>, callbacks=<span style=\"color: #0000ff; text-decoration-color: #0000ff\">None</span>, plot=<span style=\"color: #0000ff; text-decoration-color: #0000ff\">True</span>, wandb   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">forward</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">37</span>                                                                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">34 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                        <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">35 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                        <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">36 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">forward</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>, x):                                                                   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>37 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>x = <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.cnn(x)                                                                     <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">38 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>x = x.reshape(x.shape[<span style=\"color: #0000ff; text-decoration-color: #0000ff\">0</span>], -<span style=\"color: #0000ff; text-decoration-color: #0000ff\">1</span>, x.shape[-<span style=\"color: #0000ff; text-decoration-color: #0000ff\">1</span>])                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">39 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>x = x.permute(<span style=\"color: #0000ff; text-decoration-color: #0000ff\">2</span>, <span style=\"color: #0000ff; text-decoration-color: #0000ff\">0</span>, <span style=\"color: #0000ff; text-decoration-color: #0000ff\">1</span>)                                                              <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">40 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>x, _ = <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.lstm(x)                                                                 <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/lib/python3.8/dist-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">module.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">1501</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_call_impl</span>             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1498 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">not</span> (<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._backward_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._backward_pre_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._forward_hooks   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1499 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_backward_pre_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_backward_hooks                   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1500 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_forward_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_forward_pre_hooks):                   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>1501 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> forward_call(*args, **kwargs)                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1502 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># Do not call functions when jit is used</span>                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1503 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>full_backward_hooks, non_full_backward_hooks = [], []                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1504 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>backward_pre_hooks = []                                                           <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/lib/python3.8/dist-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">container.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">217</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">forward</span>              <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">214 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># with Any as TorchScript expects a more precise type</span>                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">215 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">forward</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">input</span>):                                                              <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">216 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">for</span> module <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">in</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>:                                                                <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>217 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">input</span> = module(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">input</span>)                                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">218 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">input</span>                                                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">219 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">220 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">append</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>, module: Module) -&gt; <span style=\"color: #808000; text-decoration-color: #808000\">'Sequential'</span>:                                      <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/lib/python3.8/dist-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">module.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">1501</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_call_impl</span>             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1498 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">not</span> (<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._backward_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._backward_pre_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._forward_hooks   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1499 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_backward_pre_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_backward_hooks                   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1500 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_forward_hooks <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> _global_forward_pre_hooks):                   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>1501 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> forward_call(*args, **kwargs)                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1502 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># Do not call functions when jit is used</span>                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1503 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>full_backward_hooks, non_full_backward_hooks = [], []                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1504 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>backward_pre_hooks = []                                                           <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/lib/python3.8/dist-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">conv.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">463</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">forward</span>                   <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 460 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   │   │   </span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.padding, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.dilation, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.groups)                         <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 461 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                      <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 462 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">forward</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">input</span>: Tensor) -&gt; Tensor:                                           <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 463 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._conv_forward(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">input</span>, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.weight, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.bias)                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 464 </span>                                                                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 465 </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">class</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00; text-decoration: underline\">Conv3d</span>(_ConvNd):                                                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 466 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #ff0000; text-decoration-color: #ff0000\">__doc__</span> = <span style=\"color: #808000; text-decoration-color: #808000\">r\"\"\"Applies a 3D convolution over an input signal composed of several inpu</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/lib/python3.8/dist-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">conv.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">459</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_conv_forward</span>             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 456 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> F.conv2d(F.pad(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">input</span>, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._reversed_padding_repeated_twice, mode=<span style=\"color: #00ffff; text-decoration-color: #00ffff\">sel</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 457 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   │   │   │   </span>weight, bias, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.stride,                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 458 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   │   │   │   </span>_pair(<span style=\"color: #0000ff; text-decoration-color: #0000ff\">0</span>), <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.dilation, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.groups)                         <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 459 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> F.conv2d(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">input</span>, weight, bias, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.stride,                                 <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 460 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   │   │   </span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.padding, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.dilation, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.groups)                         <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 461 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                      <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 462 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">forward</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">input</span>: Tensor) -&gt; Tensor:                                           <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n<span style=\"color: #800000; text-decoration-color: #800000\">╰──────────────────────────────────────────────────────────────────────────────────────────────────╯</span>\n<span style=\"color: #ff0000; text-decoration-color: #ff0000; font-weight: bold\">RuntimeError: </span>Input type <span style=\"font-weight: bold\">(</span>torch.cuda.FloatTensor<span style=\"font-weight: bold\">)</span> and weight type <span style=\"font-weight: bold\">(</span>torch.FloatTensor<span style=\"font-weight: bold\">)</span> should be the same\n</pre>\n",
          "text/plain": "\u001b[31m╭─\u001b[0m\u001b[31m──────────────────────────────\u001b[0m\u001b[31m \u001b[0m\u001b[1;31mTraceback \u001b[0m\u001b[1;2;31m(most recent call last)\u001b[0m\u001b[31m \u001b[0m\u001b[31m───────────────────────────────\u001b[0m\u001b[31m─╮\u001b[0m\n\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/lib/python3.8/dist-packages/torchkeras/\u001b[0m\u001b[1;33mkerascallbacks.py\u001b[0m:\u001b[94m243\u001b[0m in \u001b[92m__init__\u001b[0m              \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m240 \u001b[0m\u001b[2m│   │   \u001b[0mdisplay(\u001b[96mself\u001b[0m.out)                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m241 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mif\u001b[0m model \u001b[95mis\u001b[0m \u001b[95mnot\u001b[0m \u001b[94mNone\u001b[0m:                                                              \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m242 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[94mwith\u001b[0m \u001b[96mself\u001b[0m.out:                                                                 \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m243 \u001b[2m│   │   │   │   \u001b[0m\u001b[96mself\u001b[0m.display_fn(model)                                                     \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m244 \u001b[0m\u001b[2m│   \u001b[0m                                                                                       \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m245 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mon_fit_start\u001b[0m(\u001b[96mself\u001b[0m,model: \u001b[33m'\u001b[0m\u001b[33mKerasModel\u001b[0m\u001b[33m'\u001b[0m):                                            \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m246 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mpass\u001b[0m                                                                               \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m in \u001b[92mdisplay_fn\u001b[0m:\u001b[94m6\u001b[0m                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 3 \u001b[0m\u001b[2m│   \u001b[0mright = \u001b[94mTrue\u001b[0m                                                                            \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 4 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mwhile\u001b[0m right:                                                                            \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 5 \u001b[0m\u001b[2m│   │   \u001b[0mimage, target, input_length, label_length = ds_test[\u001b[94m0\u001b[0m]                              \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 6 \u001b[2m│   │   \u001b[0moutput = model(image.unsqueeze(\u001b[94m0\u001b[0m).cuda())                                           \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 7 \u001b[0m\u001b[2m│   │   \u001b[0moutput_argmax = output.detach().permute(\u001b[94m1\u001b[0m, \u001b[94m0\u001b[0m, \u001b[94m2\u001b[0m).argmax(dim=-\u001b[94m1\u001b[0m)                     \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 8 \u001b[0m\u001b[2m│   │   \u001b[0m                                                                                    \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 9 \u001b[0m\u001b[2m│   │   \u001b[0mright = (decode_target(target) == decode(output_argmax[\u001b[94m0\u001b[0m]))                         \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/lib/python3.8/dist-packages/torch/nn/modules/\u001b[0m\u001b[1;33mmodule.py\u001b[0m:\u001b[94m1501\u001b[0m in \u001b[92m_call_impl\u001b[0m             \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1498 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mif\u001b[0m \u001b[95mnot\u001b[0m (\u001b[96mself\u001b[0m._backward_hooks \u001b[95mor\u001b[0m \u001b[96mself\u001b[0m._backward_pre_hooks \u001b[95mor\u001b[0m \u001b[96mself\u001b[0m._forward_hooks   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1499 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[95mor\u001b[0m _global_backward_pre_hooks \u001b[95mor\u001b[0m _global_backward_hooks                   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1500 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[95mor\u001b[0m _global_forward_hooks \u001b[95mor\u001b[0m _global_forward_pre_hooks):                   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m1501 \u001b[2m│   │   │   \u001b[0m\u001b[94mreturn\u001b[0m forward_call(*args, **kwargs)                                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1502 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[2m# Do not call functions when jit is used\u001b[0m                                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1503 \u001b[0m\u001b[2m│   │   \u001b[0mfull_backward_hooks, non_full_backward_hooks = [], []                             \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1504 \u001b[0m\u001b[2m│   │   \u001b[0mbackward_pre_hooks = []                                                           \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/lib/python3.8/dist-packages/torchkeras/\u001b[0m\u001b[1;33mkerasmodel.py\u001b[0m:\u001b[94m114\u001b[0m in \u001b[92mforward\u001b[0m                   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m111 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[96mself\u001b[0m.from_scratch = \u001b[94mFalse\u001b[0m                                                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m112 \u001b[0m\u001b[2m│   \u001b[0m                                                                                       \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m113 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mforward\u001b[0m(\u001b[96mself\u001b[0m, x):                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m114 \u001b[2m│   │   \u001b[0m\u001b[94mreturn\u001b[0m \u001b[96mself\u001b[0m.net.forward(x)                                                         \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m115 \u001b[0m\u001b[2m│   \u001b[0m                                                                                       \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m116 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mfit\u001b[0m(\u001b[96mself\u001b[0m, train_data, val_data=\u001b[94mNone\u001b[0m, epochs=\u001b[94m10\u001b[0m, ckpt_path=\u001b[33m'\u001b[0m\u001b[33mcheckpoint.pt\u001b[0m\u001b[33m'\u001b[0m,         \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m117 \u001b[0m\u001b[2m│   │   │   \u001b[0mpatience=\u001b[94m5\u001b[0m, monitor=\u001b[33m\"\u001b[0m\u001b[33mval_loss\u001b[0m\u001b[33m\"\u001b[0m, mode=\u001b[33m\"\u001b[0m\u001b[33mmin\u001b[0m\u001b[33m\"\u001b[0m, callbacks=\u001b[94mNone\u001b[0m, plot=\u001b[94mTrue\u001b[0m, wandb   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m in \u001b[92mforward\u001b[0m:\u001b[94m37\u001b[0m                                                                                    \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m34 \u001b[0m\u001b[2m│   \u001b[0m                                                                                        \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m35 \u001b[0m\u001b[2m│   \u001b[0m                                                                                        \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m36 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mforward\u001b[0m(\u001b[96mself\u001b[0m, x):                                                                   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m37 \u001b[2m│   │   \u001b[0mx = \u001b[96mself\u001b[0m.cnn(x)                                                                     \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m38 \u001b[0m\u001b[2m│   │   \u001b[0mx = x.reshape(x.shape[\u001b[94m0\u001b[0m], -\u001b[94m1\u001b[0m, x.shape[-\u001b[94m1\u001b[0m])                                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m39 \u001b[0m\u001b[2m│   │   \u001b[0mx = x.permute(\u001b[94m2\u001b[0m, \u001b[94m0\u001b[0m, \u001b[94m1\u001b[0m)                                                              \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m40 \u001b[0m\u001b[2m│   │   \u001b[0mx, _ = \u001b[96mself\u001b[0m.lstm(x)                                                                 \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/lib/python3.8/dist-packages/torch/nn/modules/\u001b[0m\u001b[1;33mmodule.py\u001b[0m:\u001b[94m1501\u001b[0m in \u001b[92m_call_impl\u001b[0m             \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1498 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mif\u001b[0m \u001b[95mnot\u001b[0m (\u001b[96mself\u001b[0m._backward_hooks \u001b[95mor\u001b[0m \u001b[96mself\u001b[0m._backward_pre_hooks \u001b[95mor\u001b[0m \u001b[96mself\u001b[0m._forward_hooks   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1499 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[95mor\u001b[0m _global_backward_pre_hooks \u001b[95mor\u001b[0m _global_backward_hooks                   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1500 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[95mor\u001b[0m _global_forward_hooks \u001b[95mor\u001b[0m _global_forward_pre_hooks):                   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m1501 \u001b[2m│   │   │   \u001b[0m\u001b[94mreturn\u001b[0m forward_call(*args, **kwargs)                                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1502 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[2m# Do not call functions when jit is used\u001b[0m                                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1503 \u001b[0m\u001b[2m│   │   \u001b[0mfull_backward_hooks, non_full_backward_hooks = [], []                             \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1504 \u001b[0m\u001b[2m│   │   \u001b[0mbackward_pre_hooks = []                                                           \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/lib/python3.8/dist-packages/torch/nn/modules/\u001b[0m\u001b[1;33mcontainer.py\u001b[0m:\u001b[94m217\u001b[0m in \u001b[92mforward\u001b[0m              \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m214 \u001b[0m\u001b[2m│   \u001b[0m\u001b[2m# with Any as TorchScript expects a more precise type\u001b[0m                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m215 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mforward\u001b[0m(\u001b[96mself\u001b[0m, \u001b[96minput\u001b[0m):                                                              \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m216 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mfor\u001b[0m module \u001b[95min\u001b[0m \u001b[96mself\u001b[0m:                                                                \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m217 \u001b[2m│   │   │   \u001b[0m\u001b[96minput\u001b[0m = module(\u001b[96minput\u001b[0m)                                                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m218 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mreturn\u001b[0m \u001b[96minput\u001b[0m                                                                       \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m219 \u001b[0m\u001b[2m│   \u001b[0m                                                                                       \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m220 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mappend\u001b[0m(\u001b[96mself\u001b[0m, module: Module) -> \u001b[33m'\u001b[0m\u001b[33mSequential\u001b[0m\u001b[33m'\u001b[0m:                                      \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/lib/python3.8/dist-packages/torch/nn/modules/\u001b[0m\u001b[1;33mmodule.py\u001b[0m:\u001b[94m1501\u001b[0m in \u001b[92m_call_impl\u001b[0m             \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1498 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mif\u001b[0m \u001b[95mnot\u001b[0m (\u001b[96mself\u001b[0m._backward_hooks \u001b[95mor\u001b[0m \u001b[96mself\u001b[0m._backward_pre_hooks \u001b[95mor\u001b[0m \u001b[96mself\u001b[0m._forward_hooks   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1499 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[95mor\u001b[0m _global_backward_pre_hooks \u001b[95mor\u001b[0m _global_backward_hooks                   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1500 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[95mor\u001b[0m _global_forward_hooks \u001b[95mor\u001b[0m _global_forward_pre_hooks):                   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m1501 \u001b[2m│   │   │   \u001b[0m\u001b[94mreturn\u001b[0m forward_call(*args, **kwargs)                                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1502 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[2m# Do not call functions when jit is used\u001b[0m                                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1503 \u001b[0m\u001b[2m│   │   \u001b[0mfull_backward_hooks, non_full_backward_hooks = [], []                             \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m1504 \u001b[0m\u001b[2m│   │   \u001b[0mbackward_pre_hooks = []                                                           \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/lib/python3.8/dist-packages/torch/nn/modules/\u001b[0m\u001b[1;33mconv.py\u001b[0m:\u001b[94m463\u001b[0m in \u001b[92mforward\u001b[0m                   \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 460 \u001b[0m\u001b[2m│   │   │   │   │   │   \u001b[0m\u001b[96mself\u001b[0m.padding, \u001b[96mself\u001b[0m.dilation, \u001b[96mself\u001b[0m.groups)                         \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 461 \u001b[0m\u001b[2m│   \u001b[0m                                                                                      \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 462 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mforward\u001b[0m(\u001b[96mself\u001b[0m, \u001b[96minput\u001b[0m: Tensor) -> Tensor:                                           \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 463 \u001b[2m│   │   \u001b[0m\u001b[94mreturn\u001b[0m \u001b[96mself\u001b[0m._conv_forward(\u001b[96minput\u001b[0m, \u001b[96mself\u001b[0m.weight, \u001b[96mself\u001b[0m.bias)                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 464 \u001b[0m                                                                                          \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 465 \u001b[0m\u001b[94mclass\u001b[0m \u001b[4;92mConv3d\u001b[0m(_ConvNd):                                                                    \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 466 \u001b[0m\u001b[2m│   \u001b[0m\u001b[91m__doc__\u001b[0m = \u001b[33mr\u001b[0m\u001b[33m\"\"\"\u001b[0m\u001b[33mApplies a 3D convolution over an input signal composed of several inpu\u001b[0m  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/lib/python3.8/dist-packages/torch/nn/modules/\u001b[0m\u001b[1;33mconv.py\u001b[0m:\u001b[94m459\u001b[0m in \u001b[92m_conv_forward\u001b[0m             \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 456 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[94mreturn\u001b[0m F.conv2d(F.pad(\u001b[96minput\u001b[0m, \u001b[96mself\u001b[0m._reversed_padding_repeated_twice, mode=\u001b[96msel\u001b[0m  \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 457 \u001b[0m\u001b[2m│   │   │   │   │   │   │   \u001b[0mweight, bias, \u001b[96mself\u001b[0m.stride,                                    \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 458 \u001b[0m\u001b[2m│   │   │   │   │   │   │   \u001b[0m_pair(\u001b[94m0\u001b[0m), \u001b[96mself\u001b[0m.dilation, \u001b[96mself\u001b[0m.groups)                         \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 459 \u001b[2m│   │   \u001b[0m\u001b[94mreturn\u001b[0m F.conv2d(\u001b[96minput\u001b[0m, weight, bias, \u001b[96mself\u001b[0m.stride,                                 \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 460 \u001b[0m\u001b[2m│   │   │   │   │   │   \u001b[0m\u001b[96mself\u001b[0m.padding, \u001b[96mself\u001b[0m.dilation, \u001b[96mself\u001b[0m.groups)                         \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 461 \u001b[0m\u001b[2m│   \u001b[0m                                                                                      \u001b[31m│\u001b[0m\n\u001b[31m│\u001b[0m   \u001b[2m 462 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mforward\u001b[0m(\u001b[96mself\u001b[0m, \u001b[96minput\u001b[0m: Tensor) -> Tensor:                                           \u001b[31m│\u001b[0m\n\u001b[31m╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n\u001b[1;91mRuntimeError: \u001b[0mInput type \u001b[1m(\u001b[0mtorch.cuda.FloatTensor\u001b[1m)\u001b[0m and weight type \u001b[1m(\u001b[0mtorch.FloatTensor\u001b[1m)\u001b[0m should be the same\n"
         },
         "metadata": {},
         "output_type": "display_data"
        }
       ]
      }
     },
     "510be2636c4c44a38286697b18de6e4f": {
      "model_module": "@jupyter-widgets/output",
      "model_module_version": "1.0.0",
      "model_name": "OutputModel",
      "state": {
       "layout": "IPY_MODEL_efacf291a20045beb309c934165cf437",
       "outputs": [
        {
         "name": "stdout",
         "output_type": "stream",
         "text": "gt: 8CGQ   pred: 8CGQ\ngt: K550   pred: K550\ngt: ZS3S   pred: ZS3S\ngt: LYE7   pred: LYE7\ngt: E1VQ   pred: E1VQ\ngt: YZH6   pred: YZH6\ngt: NZSH   pred: NZSH\ngt: B88R   pred: B88R\ngt: P4EM   pred: P4EM\ngt: GLJB   pred: GLJB\ngt: SBHF   pred: SBHF\ngt: AR2K   pred: AR2K\ngt: 2G2X   pred: 2G2X\ngt: 2LH6   pred: 2LH6\ngt: 3UMV   pred: 3UMV\ngt: VZOL   pred: VZ0L\n"
        },
        {
         "data": {
          "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMAAAABACAIAAADDDu+IAAAaq0lEQVR4nO1923NbR5rf953T59INgARxEUmLskT5uvZYkuWLbGtmbM9Y2dnsZJxNOamtfdhNqjKVPM0+7fwRu0+7L5uUq5LMVqqySaYq603V7FRkj8Yey5YvkiyNLVm+0JJNihSJCwEC3efS53x5OAAEkQAFULzZNb+CWBB4Guw+59ffvbuxGmv4RuDvH7svefPHZz/d2Z70w+7v4QaA3wwC+cvVlWszM6/8/OCLL2X2H3SyY4P8apf08GuNIQikmytaSQBgqQzjYit7tRH4y9XkzVr2fPDXfwUAIwemD//kpzv45Hr2cNd+7YBgg1+qlZz/zS+jiBUOP85cxoRgXDAxunWdGwr9bt/MKz9P3hx88aVt7E4PrP+AN8aDbsEGsAOCzRj8Uq1k0AgaX3351amTs78+OX/6HxKBtMtx8MWXRg5MH/nzv9jNiiPhwYW/+cuVazMdJg3Y6oO//qv61S8682SbMYQEQjQNMLTnh57nUwhw9+4nkJMdAzh4+Cc/hR2S8IOgW8/OvPLzpLcDYsfl6xAEAsAYsP3WBrQB7C3o0iajmzcdxu82G27DPDj44ks7a5gPQSACADTo5gdGh05fC2glvXKpdOFc9v4H3VyBiRTjfKc71cLGeLAb5OswEgiRKAYAIAAE/LqxR0t5/cybsWzOvXbV2TNReORRN59nXOy4NEp48PC//w9ayXClEjZWGBeJjzJY253EUCqMwDQBABAAkQCGssF3BFpKrZRWTa1k6cI5kSssfXUt8lToKVUu8Vxh4thxN7/zGs3JjpFeWXzvVb+GgIJxMfHUcTdf2PGO3RZDEQggjgABCAgJTAZIt2+yBUjECQAwIYBAK6mVBMBk4gJQ57fBysrCW6e95aU48AGo8dWXgEgEofK08iKpZr1X9z3+NObzjAtKpXZkOAAQe15Qrcjrc2oZADhzna+U3Pf8id3PoWEIRARESERAQETkJXbR9kNLufDO6VCq4uFHgeLSB+e0pwAMlkoVDh0BgNKFcwSUf/iQX6ksvHcaEUzHQWYiASECIAIQYKiaUIbZX5/c76ThuecY7NkpDkVKNa7MxEEOsA4EoVJUWpw/c3rf8ye+QQQCAEwog0AxRCEkJtG2IzGHvUrZqyxhHEfNZuj7gGA53CstAkCoJAI0r8815770qhWKKb13ijELEDDpPgEiAWEom7iycs2p7z91Cn74Q3OHCKSV59VV6IcAFgEBJlK2sfsDJcPZQBRHyTsEMMC8rQBKTBAAYIJv1kxKFFYoZSiboWwCtVkNGHoqVE1AJEQA0I1Gc3EhbDbsVCbWEbgIQJRQqKV8CQkCA8FTV73aVKXkpNPWTsz4mGINQHFM1LmnuEMGwnAYSoUBErQnL2B8+/FppebfOh3rYOLp42gy096EuJGWsnTx/M2piUCEgIgJHbq8Q2pRJRE6SRiiZf8jIFBMCASIQKFpxr7/5Xtv3Z3OYN5grnvn/RwOaAAiGAZEUWtUEJNhbuJf2HDKbP2Gw7lRXZRBbdzGj9ee51Uqcn62MXutcumjTZHGWkmvUvLKJa1kiylELcXU6hckZAIiAEIgg5mGwZCZAASILJ1yxydGJ/e5hT1MpBExaaOV8suV+TNvByv1WG93hQIy03QcwzQ7OtZKpU3ubpbY3liqZJCGwxAomSLJJEbDRAa4XnOtZOXyR9qTQW25euXD5vz1O+eQVqp08byWEiEGIgRCBDQYGiYZiZQhAAKKAQGJECKDMbdQMJgFQIy75sTk1He/N/nCiannT/DCBOOcWq0gUkotLlRnZhK1u62g2HSdRH8lw2Bc7Dl8lIlNINCGU2aDNBzSiG4DCU1iiXboB61U2Kj7y1W/Xq1e+TDyvbtP/GFqci/j/E5CwFZq1M6knFQKTAYWoyA0TcenmEU+AUAcalXTMkrmhmHaBjNMzpExgBgF33fk8dTUPst1tVJ7OZ89dZLabj9RrGVj5cqlkclJk1km30ZFFumwuoDRTclncrF+kHMolbThVMltGw5HIAQDABAIY4pCjXFfLyxSKmo2g0opqC+XL34AAEtn3zWYk733gfEnj22YQIyL8SePBcsPyBsLYnzcsO04DOVSyR0Z9eev2xPjYbO++NYbGpoAYDDbdIQ9ljXMZJjo2py5ruW6AMA4d/P5yW8/O3vqZBwEsdYIEAeBv1wunz9nf+fZ7SQQRRo9L4pCQqNtKKxnYg5bxbHhlNltGw4rgaj1z2I2xWhZPS/SSgblcvnM21rK+hefJx/y8UmvUlKl0TtREIn0stKZ1NQ+ADBtOwqC1N59se+PHDgQ1GvN+fnYMAkAgJCZfGICAJCxtnqLocteYly4uUJm+t5ANeN6jcAAgFA2mzeueuX7Tdfcnmon7Xna872uyWjFsaUj1md+rs3ery+NNpwyG6ThBo1og1ns4D3IevNPS3XjzFuN6lKo5Mj0PXZmpPDIETsz4mSzVmrkzlOYpm0nr857K5OJKPabjdq1z0KpEkPCYCaaBpot9qDtkEgZt3qCjPPU9CQJ0OgnHCOiQDYWz/3TtsVgtFJLV66ESlGXTZn31zPkuzXLIAaykx1LXsP27bYNh0xmtaMUaNsQx9jLLdeq6VWWZKUUyIZhW1ZmJP/IESsz4mSzbj4//uRTWxFajYIgqNWuv/tWUK9RFAIQksY4ROp4xWAYZrp4l3Grb2zatjOS1dmG5drti0Er6dUg8fU2vatroZUMV5Yj2f23sMxt6F/t0KmSs0fHglp1B2vKBlVhSfgu8lSSijeYaTBm9JJAWsrShfdD2UhcCsN2EAgIGE9NPHnczeW2oohCK1m5fCkoLSVJMUBkPCXMILRGAz/UUiGiQbFRrRq02rZgXNx7+F/NLf5D4GuKCQABrEhF5Q8/4oW7tiOTQDHEOolbEQEAaYQQDd2HP92aBQA+/ruXkzc7UlM2MIGkLF04G0qZzArDdtL7poxeNpBWSitPq2YSXwEAIGJCiFzRzW1JalBL6ZXLtZlPtGwm/GaptJsvjD/+hJbN+XffiaQEAowiLasQrVYNLh8zcpS752jlo7NBw4sNCwC1lCR9raQD+U3v8GoQYUxAcSuERYCIRqzXRknW2jr+cnVna8qGk0AtWiCYtmVaVv/IsgFgtGK/AABguW7+0OFNiWr07Nvi+++FKyuxDoHAsB07m5389nOp4rhXXmSum0TPozhGPhr1skxNkc499K1w9lItNgIvTD6MSG9Dslh7nvZUqBqJ6EtEEBMpNpJfJap7el47XlM2hA1EnSRB2xVbiw7PCKhNHrJEysnnnfzqO7IpiJTyymW/WopkM8loWFxk73+Ij+UZ551MGQEQoicb1MuwMG3bHh0be+afM5FuP0cKVDNUza02g7SUix99pJXX6SQAMp4uPHKIiZuZ3XViehs2kDcFwxjRN8Ok0M++0yoxgJpdbdAUonDkia0QP9hs6qWl5TNvadUgTOZuWhTHx6bvSchqMMswLcNigBBHURxHnXzwKpipESOdMUZykFTtImoply6e07fYtuAvV5PXZg1BewpWapHyu24pWVyY3F0133a8fr4nNlBS2BG0q6FV0ysvedVqdHPWksm5m9/j5ouMb36lhJZS//qULC1ESgEAgmGJTPHRx53MSNvAJys7BkQJ7bVS7eqzHmBcgADGU5hkxzyl5S0XD55RGpxnSDEYBkBHsRIiEfi4RsbvzvVJAxEokjKSMpaqS3P1YFDL/2o2OkVbAMiEKBw6vCWue3PFL9243qiGSsUACMiESOWLbq5gtucuAQX1G50aCa3k/DtvqUpFe97aL2ScTzz8hMXdtu6FqItwg2eUhspcEkCcCMV2hMR0XUc4q8SPkx3L7D94+Cc/3VXsgQEJFEdR7aNLURS1096EvVSYVjKUnpYysToIyOKC5/JurrBFrnvp/TelJwMDAYCJFM/vKR57qvtvmbaFBjMdJ/lvJKVfqcyffb9nNNxyUzxX5Pki4yJxqkMpSxfOd7TYzCs/D2rVoFbd++wL/Xo1dOaSEnTKUIiJ1NihY90GUIKdtXX6YVAVZu2b6uitOAgi348Cv/PblvIqL2kpO7rciGPLdorfenQrrB/tKa9SXpFJvAAQ0RLp8WNPO7m82UUgxlN7Hn3G4qlknASgpYR1tVj+0FHGBQJSu/TRL5eSKN86vOnGsMYKtkzLlsw2RdoUaXMLNP5WYCACaZPYiIjMpMIGoiCoX/089rsJJG+886uFM29GSmF7KjFXZFKjW1QWrlVz8cpvQ9+PDYYAFhep/B53jaPHRNrNF1PZYtuyAaCYAg97GnEAjAs7PWKPjiFjbbNJVs+fX0W4uddfXadvGzBWqO2aIICWy1rWtKwN0nDHMRCByA9mPztLQZDMlkT83CKBpJTlulcpa0914odmSow+9dSWiB8pvXLJX7ihpUQAQmBCjB061JOpjIviM91LZCj0ZeA1Q793TpfFJETGilv+vpayWaskZtPc66/ao2P26Ng6omUoY0VLmbywExlB8FR1/vw/emrTHL3b4k5cy8FsIN9L1zD2fWrNlVumbyv07PkABrVX+lg85RTHnWJx08vUtWx45cWFN98IGxIICMESKbtYdHI5s5elZQrhFArjTx03Be8UH9745OIq/7wDwzRHyiU0TYTWEkpqNiof/hYazQFFy+DGilaqcv6mjUVABDFhXHd62PhbhI7JX7pwrn51uHpFGDASbQHjtmiSGUIMAKZth/W6ltJfrpq24VXKC++8oaW8WQ4OhABj9z2wTjpww4ikWnzzV161HClFCEgGE+nitx5dR1EyLtx83s0XtFSRkqFsuuW6VhLGeqQpDNuWU/tYveKHPiAQkJbNscVF5tiZwibHfCMpQ1+1YxBAQKbguIcfePIPt6eSpGPyB7Vq/eoXxSNHH/zTHw/1DYNJICLqWsHDbNcZG7v2y//bnL/uVcpfvfo/vPKSbgUPqVV6nBLNudlNd760kqpSUvWGVhIQMY4sx+Zjhdtm2RgXxUNtcz6KdVNGDdnTF6NUShyc7loghgi07CBQvOl+ECEBEmFLXSIC46kDj/8gndvr8m3ythKTPxE8hcOPDdt8QALFTa8ZUasay3Bs03G1ks35uau/+IW/wrRstAR+Eo/hwkil8oeODNub2yJUcvGjc6GnAA0AtERajPCJJ5+8LVNZq0I0qYBGLWXl4gdRLy1m2rY9mrVGR00u4sCPg8DTgW/ZobEl67iJIurEqQhtnrbclOVunwt28MWXnGz2wT/9sZPNOtnssM0HJVCERkRxYgIRoJ3NTr/40tzrr6lyWS4sRkGLO0nuyeJi8sln3Fx+c/2vlu28VA5lExARyOLu+HdOuLmBsmxMtHcsQNCe8spLXqUc9RJChmHkH3gYwjBYqZcvng8bzTjwN303iSQ8m2TBgIiAkDHDYAbrXee5dTj44r8uXTi799kXLv/s5XWCnz1t7cG8MMDYiJOFMkkmw8pmTdvef+IP6p9drn/xaeR7kR9Ae32NzVO2u/m7XoRKLn1wVkuJgEDEOE8Vck5h3ExlBmmeaDGrRTUKPVn97YWeASErJv/KFarVyhc/CBr1+hefQbz5y7i1ktVz5zuBDyRgnPOJvdu2ljlhw9zrr829/ureZ1/4+O9e9peX+wU/+4XXBzKikWLU3beQKAwQoP7J5ZH9016tVr86k7prygwcw7at7JjDU+Zme+9aSb9S8ivlxGcxkGzBxw4PkaNlXDDXNbkLFAMaWklZWfIqZZOLtQIsXautXP0cAIAgc+Ag0eYzSEsVBEpL2VkXaXExeuBAT19yc5FQJ6hV515/be+z3weAyz972R4dgz7Bz3X2UBuMQABmTNhOYCBg3KhVLl/2m01kzMpkRpyD9S9mACk1OeVmRtIPPrhqGt35TqJaynJ7QSoiMiFELuvkisNFbBEhDrAdUw+bjcX33tkrUgbmja7VqIbr0hNPZOe/bCqZmT5op0c6yZBNBCHEBnStjELDdox2rfcWoZs6ADD9o5eSJ/J7f/bj9avS+oXXBywoQ9MwELEdMCXtRaE3T0QEhmm7ADhyYNowTX+56teXg5V69ZNL9mjLZ7nznUQT60eVlpI8PzLLGh0be+x4R/wMSFDGhZ3J+bweKo8AQiVV6cby+XOFZ47bXQSiVMrI5djkRD46AgCm7Wz6MvVW4ZRsEiIQJcVTWkqvUmEizVx3cwOwHd4AQEKdvc9+v/OAAOC2VWn96h4HlUCtBQOULB02KIpiIkJMzJ5kaYSdztz9g39hZzJJF5NerlwDf3n58s9ehuF3kOxAK1n+4Gxrl2oh7Mzo2IOHnHxL/AxOUCZS+UceUaWFUK0A2EQUymb9xuxItcq46AghrZRWiqLYtBPBk+zWsJlGtFZy6cI5rRQSdGrctO/deO/d6scf7nnsMTdfYGL0DqXRWt5AD+oA3G7irVP3OBCBCAhIt1wsIgSkOAI0km0NAJItDNDKFdLT9zqp1PSPxro7Xb86E9SqSfWuv1wdVgK11sNXy1pKQIyDIPfgw+nJCeZy6FLPkVKXy//58E9+atpOv+nLuHDzBSdLWqVCFQJQTLHfbNYvX7LHcvZNAsnyh7/VSiVyRwOGwMKhOj0AWCplcg5EWrYKhSMpw2YzqJtB5eP8oRdyDx3eGIH68QYA1lJnQPRrMqAKo7jlh2Bnp0QCglb1HiACE2kazTEhOv3zl6sJkwqHH5t55X9n9k+XLpwdOTBdunB2qJFoJcsXz4VKAUDk+xRFn/6v//7gn/xb5nLDsbxq5dJ//U9y/npQq2Ym9n75T/944F++5GLfuCLjonj0B96vT4ZeDSgGgkjK2vysO/ul3VwBAEDwK2VVWmhnGMjkvO66m7vtC+Ni8tgzK3eNL7z9pl7xAB0wDIIYINbSi322fPGsmJgynNV1if3QTRrYPN7cfiCDXYZISIAISEBgGMk2ZdCOHjJXOLnRqaOPdd/lbiZl9k+vmhDQVnAJ+o0wKahQlbJWMgqCoFEvX/yACXHpv/ztPb//I3H/vdfePmkaVlCrGrZT+3LGmRxfeOfNqef/WV8CiVE3H7nFu7X6XMsGEMRx5DdXrr/9hskFUGwYGIVeJJutdTYAzLWnD29yTVxr3TuFjUkOWAxVoJVEMIAQ0KYYPKWvv/XGxLHjjLsWF6YQq0o6VzEG1txb2Ere3BzIYJchGqy98wi0Aqftug00mJXJ3vX08zw31nNnnbUyKfm8e8DQi09WOuOvrCycfz+ULedr5epMsnsXG82WP7tcmbtKvl/78nMrM0JA6el72ouK1ls9zTgvPny/t3hVNwJAGwhae1VBe4QAndJ6S/B0PpfJ512x+d4142Ly23+kKkvzl8565RLWddSUhEiIoaeoXJo9ddLmgqXM8WPPB2Zde6ofY6BNGtgW3twcwlBX062xEAIwAK1UOr3vATdfuG3Vc/eoVpEJet2OSEfVLz6vfXI5qFWREADTe/c35q5lDhxkQoRRhI1mFPiZ/ffUr34+Mn2vnUn87bX1xLeOmXM3l3dH40gKrcJbDeSbaQUAICRDuPkjj/czqu4wPMHEKBOjZmrkQGHCK5duvP2mH1PoebHvR0HglRcRECC2uCvnSsWHHll45zQAQDrVuUWdr9pO0twyhMEuI+haI9z1ccwIOE8VD+4bVsKvGm1PPvn1ulcpeeUyhUHyoVsopu6ainwfAMJGo9OhsQceMmzHsB0ggAGCfoyLyWf+aO7UL1W1HkrVGVpHLRMQ48JMC75/ot+KgDsPT3SrIS0lHyvWPv449BQANq/P3jwWAI3G/ELl/HvTk3cbx54yi0UmxE4xZhUGdOOp/RM7SS8EIIxN15mwLSc7ajp9MziDTNOefKrPfDp/5jSiAYDJjoGN63NdS4AhSZUTQMKqcGUFEQii5U8ue+WlVX+i+44zMeqiNXXs+dlTv8BGq6o6+cKk4AkpZkJMHDuenrq7p/hZ/4CLtQZKP3TLXS0lMhau1LVSbqHILCuJniAiEDmMVTM8U17cc/QxJ1dYWzS9IxiIQIynLNc1OU8yf+21YcREihUy1neeY5z3m/Ibm6bJkzYYa8x91clixr4/csCJfB8pbh26QASIluP4teXO1haW59949+2eT73bzAIAvbQoZheXZT00zO6VkK0fiH6p1C+EmAS3Vq7NJF9bunBuVSp7lUbuh241lHxt/erM5//nfzrZMWtklDGbEMgwEqkYENVq1fBXv7zre3/gIu6GHYAHIpDJ+Z6nvxucOumVy2ESCwZgIuXk83ufO2HmC9Sn/OBOzqFp46bSNGzbtCxWHAdApJjQYDzFhIuGkfu9b0VBUP30yth9D7D+1m7nobYkxNysaTsURRF3I9VaDsCEMDm3uIj8vkRMvsHJZgEOOtnsyrWZtcJmFTN6YpUa8perADNzr7/GuIh8PzO1n8Ig9DytFCWL5gG18mSl+tUbJ6eePeHmCjuyp2w3BpNAIuUiTj1/Yv7Mab9S1rKpPcVEevKp7/BckbnrjeFO11MSIYHJeSvfSOTkC8XDR5c/vQIEhSNHGectlxihcOQo9D+Jp2NmJbN87vWXASCz/+B9f/JnK9c+Cz3fAAMADe7eLD3bKDZsoCS3i3Hx0L/7j2LPeBxFN97+TRLF6MwlrSSVFufefePuZ39/xwk0zJGXSmopG7OfL5w5raVMT98/9d3vu7n1Nq+4w4NCvXJp4cxpf+mGmU5n73uw+ukVBJh46niHIgOeSLK2Vxf+5i/rV78AgCN//hd8z4Rp21qptgRqM3LbsfZ2aSm9Smnh7d9EzSYZBrVDJ4Qx2zO69/HnRa64pf1J3qzz4IY+dNcrz3/1//6bDsTk099L331vkk+4w070Q0JZrWT349wYaVZ1adeef7v2diU51ySN07mMkJKDQbaudnHAuzQ0gbSseaX56se/KR79oVuY3Iyu7gB29qDa3Y/BDyseeptfJkbdAkx++9/caR93FL/jzW0xoPH6DTk3fkD8TvAMiMG1/DeWQD13g9u1ps8uxICTbbcfObgxrK0A3w0nZH+9MOAiOAO6uPbNQD+u7M4dvr7uMP7+sfuGPcFl96MnV3bnDl9fdxjQdbu/MVjLlV27w9fXHfi3j07/8dlPd7obv8PXFf8fUWM3NBpiQe8AAAAASUVORK5CYII=",
          "text/plain": "<PIL.Image.Image image mode=RGB size=192x64>"
         },
         "metadata": {},
         "output_type": "display_data"
        }
       ]
      }
     },
     "5698b97586c1475e857f3f74dbe1a9f0": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "6a8e4a2263184121b6ce6dd5619e7112": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "77120b4441ef4d7a97a8177aa8032cff": {
      "model_module": "@jupyter-widgets/output",
      "model_module_version": "1.0.0",
      "model_name": "OutputModel",
      "state": {
       "layout": "IPY_MODEL_840a34942db44bb38f69edd6dc6524c2"
      }
     },
     "840a34942db44bb38f69edd6dc6524c2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "efacf291a20045beb309c934165cf437": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
